Introduction

The psychological construct of self-regulation, or related concepts of impulsivity, self-control, inhibition and others correlate with problematic real-world behaviors such as disordered eating (Mobbs et al., 2010, Verbeken et al., 2009, Nederkoorn et al. 2006a, Nederkoorn et al., 2006b), gambling (Lawrence et al., 2009, Fuentes et al., 2006, Alessi and Petry, 2003), drug addiction (Coffey et al., 2003, de Wit, Crean and John, 2000, Sher, Bartholow and Wood, 2000, Kirby, Petry and Bickel, 1999) or bad financial decisions (Meier and Sprenger, 2015, Meier and Sprenger, 2013, Meier and Sprenger 2012). In these lines of work measures of self-regulation are used as behavioral assays for individual difference analyses. An underlying assumption of treating behavioral measures as reflective of person-specific characteristics is that the measures of self-regulation are stable across time. In other words, that they have high test-retest reliability (or high between- and low within-subject variability). This assumption is not extensively and equally tested in the literature for all measures of self-regulation.
Self-regulation measures can be grouped in two broad categories: Surveys and cognitive tasks. While the former typically goes through some form of psychometric testing often filtering out items with low test-retest reliability this is less frequently true for the latter. Though some empirical results on test-retest reliabilities of certain cognitive task measures are reported in pockets of the literature an exhaustive summary or systematic elimination of tasks or measures based on stability across time does not seem to exist. Test-retest reliability is, however, crucial for individual difference analyses (Hedge, Powell and Sumner, 2017).
To answer the question of reliability of self-regulation measures comprehensively we created a large battery consisting of both cognitive task and survey measures. In this paper we make two contributions: First we provide a review of these measures along with their reported reliabilities when available. Then we report results of a large study we conducted where we collected retest reliability data using all of these measures. This effort not only fills in a notable gap in the literature in clarifying which measures of self-regulation are stable across time but also provides guidance on factors to pay attention to when constructing new tasks for individual difference measures.

Methods

Data collection

This sample consists of data for 150 subjects of the original sample of 522 that has completed the initial battery of 37 cognitive tasks, 23 surveys and 3 surveys on demographics. Details of the original sample as well as quality control (qc) procedures are described elsewhere (Eisenberg et al., 2017). Invited participants were chosen randomly and only subsets of them were invited for a given batch (instead of opening the battery to all qualified subjects) with the intention to avoid a potential oversampling and bias towards “high self regulators”.

workers = read.csv(paste0(retest_data_path,'Local/User_717570_workers.csv'))
workers = workers %>% 
  group_by(Worker.ID) %>%
  mutate(Retest_worker=ifelse(sum(CURRENT.RetestWorker,CURRENT.RetestWorkerB2,CURRENT.RetestWorkerB3,CURRENT.RetestWorkerB4,CURRENT.RetestWorkerB5,na.rm=T)>0,1,0)) %>%
  ungroup()

worker_counts <- fromJSON(paste0(retest_data_path,'/Local/retest_worker_counts.json'))

worker_counts = as.data.frame(unlist(worker_counts))
names(worker_counts) = "task_count"

In total 242 participants were invited, 175 began the battery, 157 completed the battery and 150 provided data that passed qc for both time points. Our target sample size was determined in advance of data collection and data collection continued until this number of participants with data that survived qc was reached.

disc_comp_date = read.csv(paste0(retest_data_path,'Local/discovery_completion_dates.csv'), header=FALSE)
val_comp_date = read.csv(paste0(retest_data_path,'Local/validation_completion_dates.csv'), header=FALSE)
test_comp_date = rbind(disc_comp_date, val_comp_date)
rm(disc_comp_date, val_comp_date)
retest_comp_date = read.csv(paste0(retest_data_path,'Local/retest_completion_dates.csv'), header=FALSE)
comp_dates = merge(retest_comp_date, test_comp_date, by="V1")
names(comp_dates) <- c("sub_id", "retest_comp", "test_comp")
comp_dates$retest_comp = as.Date(comp_dates$retest_comp)
comp_dates$test_comp = as.Date(comp_dates$test_comp)
comp_dates$days_btw = with(comp_dates, retest_comp-test_comp)

Data collection took place on average 115 number of days after the completion of the initial battery with a range of 60 to 228 days.

rm(test_comp_date, retest_comp_date, comp_dates)

Demographics

test_demog <- read.csv(paste0(retest_data_path, '/t1_data/demographic_health.csv'))

retest_demog <- read.csv(paste0(retest_data_path, 'demographic_health.csv'))

retest_demog = retest_demog[retest_demog$X %in% test_demog$X,]

names(test_demog)[which(names(test_demog) == 'X')] <-'sub_id'
names(retest_demog)[which(names(retest_demog) == 'X')] <-'sub_id'

summary(test_demog %>%
          select(Sex, Age))
      Sex            Age      
 Min.   :0.00   Min.   :21.0  
 1st Qu.:0.00   1st Qu.:28.0  
 Median :1.00   Median :33.0  
 Mean   :0.52   Mean   :34.1  
 3rd Qu.:1.00   3rd Qu.:38.8  
 Max.   :1.00   Max.   :59.0  
summary(retest_demog %>%
          select(Sex, Age))
      Sex             Age      
 Min.   :0.000   Min.   :21.0  
 1st Qu.:0.000   1st Qu.:29.0  
 Median :1.000   Median :33.0  
 Mean   :0.527   Mean   :34.5  
 3rd Qu.:1.000   3rd Qu.:38.8  
 Max.   :1.000   Max.   :60.0  

Literature

One of the major contributions of this project is a comprehensive literature review of the retest reliabilities of the surveys and tasks that were used. We reviewed the literature on a measure (as opposed to task level) paying attention to differences in sample size, the delay between the two measurements as well as the statistic that was used to assess reliabilities (e.g. Spearman vs. Pearson correlations). Here we present a table and a visualization summarizing our findings. References mentioned in the table below can be found here.

lit_review <- read.csv('/Users/zeynepenkavi/Dropbox/PoldrackLab/SRO_Retest_Analyses/input/lit_review_figure.csv')

lit_review
lit_review = lit_review %>%
  separate(dv, c("task_group", "var"), sep="\\.",remove=FALSE,extra="merge") %>%
  mutate(task_group = factor(task_group, levels = task_group[order(task)]),
         type = as.character(type)) %>%
  mutate(task_group = gsub("_", " ", task_group),
         var = gsub("_", " ", var)) %>%
  arrange(task_group, raw_fit, var) %>%
  mutate(task_group = ifelse(task_group == "psychological refractory period two choices", "psychological refractory period", ifelse(task_group == "angling risk task always sunny", "angling risk task",task_group))) %>%
  mutate(task_group = gsub("survey", "", task_group)) %>%
  select(-measure_description)
Warning in `levels<-`(`*tmp*`, value = if (nl == nL) as.character(labels)
else paste0(labels, : duplicated levels in factors are deprecated
tmp = lit_review[duplicated(lit_review$reference)==FALSE,]

nrow(tmp)
[1] 154
sum(tmp$sample_size)
[1] 17550
nrow(lit_review)
[1] 583
rm(tmp)

Measure level plot

lit_review = lit_review %>%
  select(-reference)

Because this plot is difficult to digest we summarize it on a task level to give a general sense of the main takeaways. This plot naturally disregards much of the fine grained information.

p1_t_legend <- lit_review %>%
  filter(task == 'task') %>%
ggplot(aes(y = factor(task_group, levels=rev(unique(task_group))), x = retest_reliability)) + 
  geom_point(aes(size=sample_size, shape = type), color='black')+
  theme(axis.text.y = element_text(size=43),
        legend.position = 'bottom',
        axis.text.x = element_text(size=23),
        axis.title.x = element_text(size=30), 
        legend.text = element_text(size=20), 
        legend.key.width = unit(0.75, "inches"), 
        legend.title = element_text(size=28),
        legend.spacing.x = unit(0.5, "inches")) + 
  guides(size = guide_legend(override.aes = list(size=c(9,18,28))),
         shape = guide_legend(override.aes = list(size=18)))+
  xlab("Retest Reliability")+
  ylab("")+
  scale_x_continuous(limits = c(-0.25,1), breaks=c(-0.25, 0, 0.25, 0.5, 0.75, 1))+
  scale_y_discrete(labels = function(x) str_wrap(x, width = 10))+
  scale_shape_manual(breaks = sort(lit_review$type), values = c(15, 16, 17, 3), name="Type")+
  scale_size_continuous(name = "Sample Size")+
  geom_vline(xintercept = 0, color = "red", size = 1)

p1_t <- lit_review %>%
  filter(task == 'task') %>%
ggplot(aes(y = factor(task_group, levels=rev(unique(task_group))), x = retest_reliability)) + 
  geom_point(aes(size=sample_size, shape = type), color='#00BFC4')+
  theme(axis.text.y = element_text(size=43),
        legend.position = 'none',
        axis.text.x = element_text(size=23),
        axis.title.x = element_text(size=30)) + 
  xlab("Retest Reliability")+
  ylab("")+
  scale_x_continuous(limits = c(-0.25,1), breaks=c(-0.25, 0, 0.25, 0.5, 0.75, 1))+
  scale_y_discrete(labels = function(x) str_wrap(x, width = 10))+
  scale_shape_manual(breaks = sort(lit_review$type), values = c(15, 16, 17, 3))+
  scale_size_continuous(range = c(5, 35))+
  geom_vline(xintercept = 0, color = "red", size = 1)

p2_t <- lit_review %>%
  filter(task == 'survey') %>%
ggplot(aes(y = factor(task_group, levels=rev(unique(task_group))), x = retest_reliability)) + 
  geom_point(aes(size=sample_size, shape = type), color='#F8766D')+
  theme(axis.text.y = element_text(size=43),
        legend.position = 'none',
        axis.text.x = element_text(size=23),
        axis.title.x = element_text(size=30)) + 
  xlab("Retest Reliability")+
  ylab("")+
  scale_x_continuous(limits = c(-0.25,1), breaks=c(-0.25, 0, 0.25, 0.5, 0.75, 1))+
  scale_y_discrete(labels = function(x) str_wrap(x, width = 10))+
  scale_shape_manual(breaks = sort(lit_review$type), values = c(15, 16, 17, 3))+
  scale_size_continuous(range = c(5, 35))+
  geom_vline(xintercept = 0, color = "red", size = 1)

mylegend<-g_legend(p1_t_legend)

p3_t <- arrangeGrob(arrangeGrob(p1_t, p2_t, nrow=1), mylegend, nrow=2,heights=c(10, 1))

ggsave('Lit_Review_Plot_t.jpg', plot = p3_t, device = "jpeg", path = "/Users/zeynepenkavi/Dropbox/PoldrackLab/SRO_Retest_Analyses/output/figures/", width = 27, height = 48, units = "in", limitsize = FALSE, dpi = 72)
rm(p1_t, p2_t, p3_t, mylegend, p1_t_legend)

An interactive version of this plot could be find here

Takeaways from this review are:
- Survey measures have been reported to higher reliability compared to task measures
- Survey measures have less variability in the reported reliabiltiy estimates compared to task measures

Loading datasets

The variables included in this report are:
- meaningful variables (includes only hdddm parameters)
- EZ diffusion parameters
- Raw RT and Accuracy measures
- Variables found in the literature (for comparison)

Load time 1 data

test_data <- read.csv(paste0(retest_data_path,'t1_data/variables_exhaustive.csv'))

test_data <- test_data[,names(test_data) %in% retest_report_vars]

test_data$X <- as.character(test_data$X)
names(test_data)[which(names(test_data) == 'X')] <-'sub_id' 

For reference here are the variables that are not included in the analyses of the remainder of this report because they were not of theoretical interest in factor structure analyses of this data so far. These include drift diffusion and other model parameters for specific conditions within a task; survey variables that are not part of the dependant variables for that survey in the literature and demographics (these are saved for prediction analyses).

test_data2 <- read.csv(paste0(retest_data_path, 't1_data/variables_exhaustive.csv'))

df <- data.frame(names(test_data2)[which(names(test_data2) %in% names(test_data) == FALSE)])
names(df) = c('vars')

df

Load time 2 data

retest_data <- read.csv(paste0(retest_data_path,'variables_exhaustive.csv'))

retest_data <- retest_data[,names(retest_data) %in% retest_report_vars]

retest_data$X <- as.character(retest_data$X)
names(retest_data)[which(names(retest_data) == 'X')] <-'sub_id' 
retest_data = retest_data[retest_data$sub_id %in% test_data$sub_id,]

Replace HDDM parameters in t1 data

Since HDDM parameters depend on the sample on which they are fit we refit the model on t1 data for the subjects that have t2 data. Here we replace the HDDM parameters in the current t1 dataset with these refitted values.

hddm_refits <- read.csv(paste0(retest_data_path,'t1_data/hddm_refits_exhaustive.csv'))

hddm_refits = hddm_refits[,names(hddm_refits) %in% retest_report_vars]

hddm_refits$X <- as.character(hddm_refits$X)
names(hddm_refits)[which(names(hddm_refits) == 'X')] <-'sub_id' 

#For later comparison of whether fitting the DDM parameters on full or retest sample makes a big difference
test_data_full_sample_hddm <- test_data

#drop hddm columns from test_data
test_data = cbind(test_data$sub_id, test_data[,names(test_data) %in% names(hddm_refits) == FALSE])

#fix naming before merging
names(test_data)[which(names(test_data) == 'test_data$sub_id')] <-'sub_id'

#merge hddm refits to test data
test_data = merge(test_data, hddm_refits, by="sub_id")

Results

Data quality checks

Demographics reliability

Point estimates of reliability for the demographic variabels.

numeric_cols = c()

for(i in 1:length(names(test_demog))){
  if(is.numeric(test_demog[,i])){
    numeric_cols <- c(numeric_cols, names(test_demog)[i])
  }
}

demog_rel_df <- data.frame(spearman = rep(NA, length(numeric_cols)),
                     icc = rep(NA, length(numeric_cols)),
                     pearson = rep(NA, length(numeric_cols)))

row.names(demog_rel_df) <- numeric_cols

for(i in 1:length(numeric_cols)){
  demog_rel_df[numeric_cols[i], 'spearman'] <- get_spearman(numeric_cols[i], t1_df = test_demog, t2_df = retest_demog) 
  demog_rel_df[numeric_cols[i], 'icc'] <- get_icc(numeric_cols[i], t1_df = test_demog, t2_df = retest_demog)
  demog_rel_df[numeric_cols[i], 'pearson'] <- get_pearson(numeric_cols[i], t1_df = test_demog, t2_df = retest_demog)
}

demog_rel_df = demog_rel_df %>%
  mutate(var = row.names(.)) %>%
  select(var, icc, spearman, pearson) %>%
  arrange(-icc)

demog_rel_df
sjt.df(demog_rel_df %>% mutate_if(is.numeric, funs(round(., 3))), describe=F, hide.progress = TRUE, show.rownames = FALSE, file = "/Users/zeynepenkavi/Dropbox/PoldrackLab/SRO_Retest_Analyses/output/tables/demog_rel_table.doc")
var icc spearman pearson
HispanicLatino 1 1 1
Age 0.998 0.996 0.997
Sex 0.993 0.987 0.987
HouseholdIncome 0.988 0.955 0.977
ArrestedChargedLifeCount 0.958 0.829 0.92
HighestEducation 0.951 0.895 0.908
WeightPounds 0.951 0.936 0.907
ChildrenNumber 0.95 0.971 0.907
HowLongSmoked 0.949 0.877 0.904
RelationshipNumber 0.948 0.9 0.901
CigsPerDay 0.945 0.841 0.895
LifetimeSmoke100Cigs 0.944 0.894 0.894
AlcoholHowManyDrinksDay 0.936 0.871 0.881
MortgageDebt 0.93 0.833 0.87
RetirementPercentStocks 0.928 0.863 0.867
LongestRelationship 0.928 0.853 0.867
HowOftenFailedActivitiesDrinking 0.927 0.782 0.866
CoffeeCupsPerDay 0.926 0.876 0.863
RentOwn 0.926 0.862 0.862
HowOftenMemoryConcentrationProblemCannabis 0.925 0.594 0.863
SmokeEveryDay 0.916 0.802 0.846
HeightInches 0.907 0.888 0.83
DivorceCount 0.905 0.935 0.836
CannabisHowOften 0.903 0.856 0.823
EducationDebt 0.901 0.837 0.821
HowOftenDevotedTimeCannabis 0.898 0.814 0.814
MedicalProblemsDueToDrugUse 0.888 0.814 0.814
AlcoholHowOften 0.881 0.788 0.79
RelationshipStatus 0.87 0.788 0.77
HowSoonSmokeAfterWaking 0.868 0.787 0.767
RetirementAccount 0.86 0.756 0.756
AlcoholHowOften6Drinks 0.853 0.766 0.747
CreditCardDebt 0.846 0.808 0.733
RelativeFriendConcernedDrinking 0.844 0.71 0.733
CannabisPast6Months 0.84 0.725 0.725
TeaCupsPerDay 0.834 0.633 0.715
HowOftenCantStopDrinking 0.833 0.704 0.716
CaffienatedSodaCansPerDay 0.832 0.711 0.723
HowOftenGuiltRemorseDrinking 0.829 0.564 0.708
CannabisHoursStoned 0.825 0.698 0.703
DoctorVisitsLastMonth 0.815 0.351 0.701
HowOftenDrinkMorning 0.809 0.39 0.692
CannabisConsideredReduction 0.806 0.638 0.675
NeurologicalDiagnoses 0.795 0.66 0.66
HowOftenUnableRememberDrinking 0.786 0.61 0.651
FeelBadGuiltyDrugUse 0.782 0.642 0.642
TrafficAccidentsLifeCount 0.781 0.585 0.643
HowOftenCantStopCannabis 0.748 0.732 0.598
InjuredDrinking 0.734 0.626 0.592
CarDebt 0.716 0.698 0.559
DaysHalfLastMonth 0.715 0.552 0.559
Worthless 0.709 0.632 0.549
DaysPhysicalHealthFeelings 0.705 0.533 0.545
Depressed 0.689 0.561 0.527
EverythingIsEffort 0.682 0.553 0.517
Hopeless 0.678 0.579 0.516
OtherDrugs 0.658 0.492 0.492
RestlessFidgety 0.621 0.507 0.451
TrafficTicketsLastYearCount 0.621 0.438 0.45
Nervous 0.615 0.473 0.445
NeglectedFamilyDrugUse 0.488 0.342 0.342
DaysLostLastMonth 0.475 0.604 0.312
EngagedInIllegalActsToObtainDrugs 0.468 0.306 0.306
AbleToStopDrugs 0.458 0.306 0.306
BlackoutFlashbackDrugUse 0.424 0.272 0.272
HowOftenFailedActivitiesCannabis 0.423 0.636 0.378
WidthdrawalSymptoms 0.403 0.254 0.254
Last30DaysUsual 0.395 0.285 0.248
GamblingProblem 0.317 0.38 0.195
AbuseMoreThanOneDrugAtATime 0.294 0.173 0.173
HowOftenHazardousCannabis 0.057 0.369 0.029
CaffieneOtherSourcesDayMG 0 0.363 -0.033
SpouseParentsComplainDrugUse -0.033 -0.017 -0.017
OtherDebtAmount -0.083 0.005 -0.041
summary(demog_rel_df)
     var                 icc             spearman          pearson       
 Length:74          Min.   :-0.0831   Min.   :-0.0166   Min.   :-0.0406  
 Class :character   1st Qu.: 0.6793   1st Qu.: 0.5373   1st Qu.: 0.5161  
 Mode  :character   Median : 0.8323   Median : 0.7006   Median : 0.7156  
                    Mean   : 0.7417   Mean   : 0.6558   Mean   : 0.6446  
                    3rd Qu.: 0.9257   3rd Qu.: 0.8401   3rd Qu.: 0.8628  
                    Max.   : 1.0000   Max.   : 1.0000   Max.   : 1.0000  

Effect of delays between the two measurements

Due to our data collection strategy we did not strictly control for the delay between the two measurements as standard psychometrics studies measuring retest reliability might.

An individual difference measure would preferably remain stable regardless of the delay between multiple measurements.

Since we only had two measurements we could not test directly whether a measure becomes less reliable depending on the delay between the two time points (The average number of days between two measurements would be the same for all measures since reliability is a measure level metric while days between completion a subject level one).

So if you regress the average difference for each measure on average delay between two measurements you are regressing a vector with varying numbers on a vector of same values, like a t test asking if the mean of the varying column, in this case the average difference score, is different than the unique value in the single value column, i.e. the average delay between two time points (Note that this should be true in theory but in practice since for each measure there might subjects for whom the dv could not be calculated there might be >1 unique values for the average delay between the two time points). This analysis is not meaningful.

Instead of the summary metric like the retest reliability estimate for each measure we can check whether the difference score distribution for each measure depends on the delay between the two measurements. Since the difference score distribution is at subject level we can check whether the order of subjects in this distribution depends on their order in the distribution of days between completing the two tests.

Make data frame with difference between two scores for each measure for each subject. Since the scores for different measures are on different scales for comparability the difference scores are normalized (ie demeaned and dividied by the sd of the difference score distribution.) Note that the variance of the difference score distribution accounts for the variance in both time points by summing them. Normalization equates the means of each difference score distribution to 0 which would mask any meaningful change between the two time points but the analysis here does not interpret the mean of the difference score distributions but is interested in its relation to the days between completion. We check if the variables show systematic differences between the two points later.

Here we check if the difference is larger the longer the delay

numeric_cols = c()

for(i in 1:length(names(test_data))){
  if(is.numeric(test_data[,i]) & names(test_data)[i] %in% names(retest_data)){
    numeric_cols <- c(numeric_cols, names(test_data)[i])
  }
}

t1_2_difference = data.frame()

for(i in 1:length(numeric_cols)){
  tmp = match_t1_t2(numeric_cols[i],format='wide')
  tmp = tmp %>% 
  mutate(difference = scale(`2` - `1`))
  t1_2_difference = rbind(t1_2_difference, tmp)
}

t1_2_difference$difference = as.data.frame(t1_2_difference$difference)$V1

t1_2_difference = t1_2_difference %>% separate(dv, c("task", "dv2"), sep="\\.", remove=FALSE)

Add completion dates to this data frame.

retest_task_comp_times = read.csv(paste0(retest_data_path, 'Local/retest_task_completion_times.csv'))
test_task_comp_times = read.csv(paste0(retest_data_path, 'Local/test_task_completion_times.csv'))
task_comp_times = merge(retest_task_comp_times, test_task_comp_times, by=c('worker_id','task'))
rm(retest_task_comp_times, test_task_comp_times)
task_comp_times = task_comp_times %>%
  select(-X.x, -X.y) %>%
  mutate(finish_day.x = as.Date(finish_day.x),
         finish_day.y = as.Date(finish_day.y),
         days_btw = finish_day.x-finish_day.y) %>%
  rename(sub_id=worker_id)
t1_2_difference = merge(t1_2_difference, task_comp_times[,c('sub_id', 'task','days_btw')], by=c('sub_id', 'task'))

What does the distribution of differences look like: The distribution of differences between two time points for each measure

t1_2_difference %>%
  ggplot(aes(difference, alpha=dv))+
  geom_histogram(position='identity')+
  theme(legend.position = 'none')

How do the difference score distributions look like with respect to the days between completion?

t1_2_difference %>%
  ggplot()+
  geom_smooth(aes(as.numeric(days_btw), abs(difference), group=factor(dv)), method='lm', se=FALSE)+
  geom_smooth(aes(as.numeric(days_btw), abs(difference)), method='lm', color = "black", se=FALSE)+
  theme(legend.title = element_blank())+
  xlab('Days between completion')+
  ylab('Absolute Scaled difference score')

To test if the slope of the black is significant we would run a mixed effects model with a fixed effect for days between completion, random slope for each dv depending on the days between and random intercept for each dv.

Before I was using subjects as a random effect but days between the two time points for each measure depends on subj id. What varies randomly is which dv we are looking for its distribution of differences in relation to the days between the time points. So I changed the model to have fixed effect for the days between, a random slope for (dependent variables can be differentially sensitive to th effect of days between) and a random intercept for dependent variable.

Significant fixed effect suggests that on average the longer the delay the smaller the difference.

summary(lmerTest::lmer(abs(difference) ~ scale(days_btw)+(scale(days_btw) | dv), data=t1_2_difference)) 
Linear mixed model fit by REML t-tests use Satterthwaite approximations
  to degrees of freedom [lmerMod]
Formula: abs(difference) ~ scale(days_btw) + (scale(days_btw) | dv)
   Data: t1_2_difference

REML criterion at convergence: 132273

Scaled residuals: 
   Min     1Q Median     3Q    Max 
-1.127 -0.719 -0.260  0.402 15.807 

Random effects:
 Groups   Name            Variance Std.Dev. Corr
 dv       (Intercept)     0.003190 0.0565       
          scale(days_btw) 0.000149 0.0122   1.00
 Residual                 0.468366 0.6844       
Number of obs: 63454, groups:  dv, 446

Fixed effects:
                  Estimate Std. Error         df t value Pr(>|t|)    
(Intercept)        0.72218    0.00382  457.00000  189.05   <2e-16 ***
scale(days_btw)   -0.00985    0.00278 3236.00000   -3.55   0.0004 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Correlation of Fixed Effects:
            (Intr)
scl(dys_bt) 0.146 

But if I just run one mixed effects model then we don’t get a sense of the simple effects (how many of the variables this effect of the days between is significant and in which direction). I can run it separately for each dv to see if all difference score distributions are affected the same way depending on the days between completion.

get_delay_effect = function(df){
  mod = lm(abs(difference) ~ scale(days_btw), data = df)
  out = data.frame(estimate=coef(summary(mod))["scale(days_btw)","Estimate"], pval=coef(summary(mod))["scale(days_btw)","Pr(>|t|)"])
  return(out)
}

sig_days_effect = t1_2_difference %>%
  group_by(dv) %>%
  do(get_delay_effect(.)) %>%
  filter(pval<0.05)

sig_days_effect

For visualization I used to summarize the difference scores per person by looking at the average difference per task per subject and plot that against the number of days between completion. This yields multiple points for each value on x representing the average difference per task for each subject. But variables within a task could go in different directions (e.g. if patient proportion increases discount rate decreases) so this doesn’t seem like a good idea)

Instead I now plot all the data grouping by dependant variable and coloring depending on whether the difference score distribution for that variable has a significant slope when regressed over days between. The black is the main effect for the large multilevel model (this is the same plot as above colored by whether the difference score distribution for each dv has a significant slope depending on days between).

t1_2_difference %>%
  mutate(sig_days_effect = ifelse(dv %in% sig_days_effect$dv, 1, 0))%>%
  arrange(-sig_days_effect) %>%
  ggplot()+
  stat_smooth (aes(as.numeric(days_btw), abs(difference), 
                  group=factor(dv, levels=unique(dv[order(sig_days_effect)]), ordered=TRUE), 
                  color=factor(sig_days_effect, levels = c(1,0))),geom="line", alpha=0.5, method='lm')+
  geom_smooth(aes(as.numeric(days_btw), abs(difference)), method='lm', color = "black", se=FALSE)+
  theme(legend.title = element_blank())+
  xlab('Days between completion')+
  ylab('Scaled difference score')+
  scale_color_discrete(breaks=c(0,1),
                       labels=c("NS", "Sig"))

ggsave('DaysBtwEffect.jpg', device = "jpeg", path = "/Users/zeynepenkavi/Dropbox/PoldrackLab/SRO_Retest_Analyses/output/figures/", width = 6, height = 4, units = "in", limitsize = FALSE, dpi = 100)

Conclusion: The average difference between the scores of a measure if anything decreases with the increased delay. This fixed effect masks the simple effect for each dv. When looked at individually only 10 variables show a dependence on days between completion. While 7 of them show decreasing differences depending on days between 3 do show increasing differences. This suggests that the lack of stricter control over the days between completion of the measurement does not have a large effect on the stability of most measures.

effect of whether days between leads to a sig difference on reliability of that variable? plot coef of sig_day_effect over rel_df point estimate

Do the variables that show significant dependence on the delay between the completion times have systematically lower/higher reliability? Looking at the reliability point estimate over the size of the delay effect. There are very few variables to compare anyway but regardless doesn’t look conclusive/concerning.

#Create df of point estimate reliabilities
rel_df <- data.frame(spearman = rep(NA, length(numeric_cols)),
                     icc = rep(NA, length(numeric_cols)),
                     pearson = rep(NA, length(numeric_cols)),
                     partial_eta_sq = rep(NA, length(numeric_cols)),
                     sem = rep(NA, length(numeric_cols)),
                     var_subs = rep(NA, length(numeric_cols)),
                     var_ind = rep(NA, length(numeric_cols)),
                     var_resid = rep(NA, length(numeric_cols)))

row.names(rel_df) <- numeric_cols

for(i in 1:length(numeric_cols)){
  rel_df[numeric_cols[i], 'spearman'] <- get_spearman(numeric_cols[i]) 
  rel_df[numeric_cols[i], 'icc'] <- get_icc(numeric_cols[i])
  rel_df[numeric_cols[i], 'pearson'] <- get_pearson(numeric_cols[i])
  rel_df[numeric_cols[i], 'partial_eta_sq'] <- get_partial_eta(numeric_cols[i])
  rel_df[numeric_cols[i], 'sem'] <- get_sem(numeric_cols[i])
  rel_df[numeric_cols[i], 'var_subs'] <- get_var_breakdown(numeric_cols[i])$subs
  rel_df[numeric_cols[i], 'var_ind'] <- get_var_breakdown(numeric_cols[i])$ind
  rel_df[numeric_cols[i], 'var_resid'] <- get_var_breakdown(numeric_cols[i])$resid
}

rel_df$dv = row.names(rel_df)
row.names(rel_df) = seq(1:nrow(rel_df))
rel_df$task = 'task'
rel_df[grep('survey', rel_df$dv), 'task'] = 'survey'
rel_df[grep('holt', rel_df$dv), 'task'] = "task"
rel_df = rel_df %>%
  select(dv, task, spearman, icc, pearson, partial_eta_sq, sem, var_subs, var_ind, var_resid)
# row.names(rel_df) = NULL
sig_days_effect %>%
  left_join(rel_df, by='dv') %>%
  ggplot(aes(estimate, icc))+
  geom_point()

Overlapping survery questions

Some surveys have overlapping questions. Do these correlate within and across sessions?

First determine the overlapping questions.

tmp = read.csv(gzfile(paste0(retest_data_path, 'items.csv.gz')))
tmp = tmp %>% 
  filter(worker == 's005') %>% 
  select(item_ID, item_text) %>% 
  mutate(item_text = trimws(as.character(item_text))) %>%
  unite(item, c("item_ID", "item_text"), sep = "___")

comb = as.data.frame(t(combn(unique(tmp$item),2)))

duplicate_items = comb %>% 
  filter(grepl('dospert', V1)==FALSE) %>%
  filter(grepl('selection_optimization', V1)==FALSE) %>%
  filter(grepl('sensation_seeking', V1)==FALSE) %>%
  separate(V1, c("item1_ID", "item1_text"), sep="___") %>%
  separate(V2, c("item2_ID", "item2_text"), sep="___") %>%
  mutate(similarity = levenshteinSim(item1_text, item2_text)) %>%
  filter(similarity>0.8) %>%
  select(item1_ID, item2_ID, item1_text, item2_text)

duplicate_items
#surveys to read in
extract_items = c('worker',unique(with(duplicate_items, c(item1_ID, item2_ID))))

#correlations to compute:
#item1_t1 - item2_t1, 
#item1_t2 - item2_t2, 
#item1_t1 - item2_t2, 
#item1_t2 - item2_t1

duplicate_items_data_t1 = read.csv(paste0(test_data_path, 'subject_x_items.csv'))
duplicate_items_data_t2 = read.csv(paste0(retest_data_path, 'subject_x_items.csv'))

duplicate_items_data_t1 = duplicate_items_data_t1 %>%
  filter(worker %in% duplicate_items_data_t2$worker) %>%
  select(extract_items)

duplicate_items_data_t2=duplicate_items_data_t2 %>%
  filter(worker %in% duplicate_items_data_t1$worker) %>%
  select(extract_items)

duplicate_items = duplicate_items %>%
  mutate(t1_t1_cor = NA,
         t2_t2_cor = NA,
         t1_t2_cor = NA,
         t2_t1_cor = NA,
         t1_t1_polycor = NA,
         t2_t2_polycor = NA,
         t1_t2_polycor = NA,
         t2_t1_polycor = NA)

for(i in 1:nrow(duplicate_items)){
  duplicate_items$t1_t1_cor[i] = abs(cor(duplicate_items_data_t1[,c(duplicate_items$item1_ID[i])],
                                  duplicate_items_data_t1[,c(duplicate_items$item2_ID[i])]))
  
  duplicate_items$t2_t2_cor[i] = abs(cor(duplicate_items_data_t2[,c(duplicate_items$item1_ID[i])],
                                  duplicate_items_data_t2[,c(duplicate_items$item2_ID[i])]))
  
  duplicate_items$t1_t2_cor[i] = abs(cor(duplicate_items_data_t1[,c(duplicate_items$item1_ID[i])],
                                  duplicate_items_data_t2[,c(duplicate_items$item2_ID[i])]))
  
  duplicate_items$t2_t1_cor[i] = abs(cor(duplicate_items_data_t2[,c(duplicate_items$item1_ID[i])],
                                  duplicate_items_data_t1[,c(duplicate_items$item2_ID[i])]))
  
  duplicate_items$t1_t1_polycor[i] = abs(polychoric(data.frame(duplicate_items_data_t1[,c(duplicate_items$item1_ID[i])],
                      duplicate_items_data_t1[,c(duplicate_items$item2_ID[i])]))$rho[2])

  duplicate_items$t2_t2_polycor[i] = abs(polychoric(data.frame(duplicate_items_data_t2[,c(duplicate_items$item1_ID[i])],
                      duplicate_items_data_t2[,c(duplicate_items$item2_ID[i])]))$rho[2])

  duplicate_items$t1_t2_polycor[i] = abs(polychoric(data.frame(duplicate_items_data_t1[,c(duplicate_items$item1_ID[i])],
                      duplicate_items_data_t2[,c(duplicate_items$item2_ID[i])]))$rho[2])

  duplicate_items$t2_t1_polycor[i] = abs(polychoric(data.frame(duplicate_items_data_t2[,c(duplicate_items$item1_ID[i])],
                      duplicate_items_data_t1[,c(duplicate_items$item2_ID[i])]))$rho[2])
}

duplicate_items
summary(duplicate_items$t1_t1_polycor)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  0.284   0.355   0.583   0.567   0.742   0.822 
summary(duplicate_items$t2_t2_polycor)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  0.101   0.446   0.612   0.580   0.700   0.904 
summary(duplicate_items$t1_t2_polycor)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
 0.0128  0.3570  0.5710  0.5160  0.6870  0.8380 
summary(duplicate_items$t2_t1_polycor)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
 0.0631  0.4490  0.5350  0.5010  0.5830  0.8190 

Comparison to prior literature

Read in and process bootstrapped results.

process_boot_df = function(df){
  df = df %>%
  drop_na() %>%
  mutate(dv = as.character(dv),
         icc = as.numeric(as.character(icc)),
         spearman = as.numeric(as.character(spearman)),
         pearson = as.numeric(as.character(pearson)),
         eta_sq = as.numeric(as.character(eta_sq)),
         sem = as.numeric(as.character(sem)),
         partial_eta_sq = as.numeric(as.character(partial_eta_sq)),
         omega_sq = as.numeric(as.character(omega_sq)),
         var_subs = as.numeric(as.character(var_subs)),
         var_ind = as.numeric(as.character(var_ind)),
         var_resid = as.numeric(as.character(var_resid)),
         F_time = as.numeric(as.character(F_time)),
         p_time = as.numeric(as.character(p_time)),
         df_time = as.numeric(as.character(df_time)),
         df_resid = as.numeric(as.character(df_resid)))
  return(df)} 

boot_df <- read.csv(gzfile(paste0(retest_data_path,'bootstrap_merged.csv.gz')))
Warning in scan(file = file, what = what, sep = sep, quote = quote, dec =
dec, : embedded nul(s) found in input
boot_df = process_boot_df(boot_df)

boot_df = boot_df[boot_df$dv %in% retest_report_vars,]

# Check if you have all variables bootstrapped
# retest_report_vars[which(retest_report_vars %in% boot_df$dv==FALSE)]

# Boot df contains hddm parameters fit on the full sample in the t1 data
# refits_bootstrap_merged.csv.gz contains bootstrapped reliabilities 

refit_boot_df = read.csv(gzfile(paste0(retest_data_path,'refits_bootstrap_merged.csv.gz')))
Warning in scan(file = file, what = what, sep = sep, quote = quote, dec =
dec, : embedded nul(s) found in input
refit_boot_df = process_boot_df(refit_boot_df)

fullfit_boot_df = boot_df[as.character(boot_df$dv) %in% unique(as.character(refit_boot_df$dv)),]

boot_df = boot_df[!as.character(boot_df$dv) %in% unique(as.character(refit_boot_df$dv)),]

boot_df = rbind(boot_df, refit_boot_df)

rm(refit_boot_df)

Summarize bootstrapped results and merge to lit review data

var_boot_df = boot_df %>%
  group_by(dv) %>%
  summarise(mean_icc = mean(icc),
            mean_pearson = mean(pearson))

rel_comp = lit_review %>%
  left_join(var_boot_df, by = 'dv')
Warning: Column `dv` joining factor and character vector, coercing into
character vector

Here’s what our data looks like: (583 data points for 171 measures)

rel_comp

Lit review results

Distribution of reliabilities, sample sizes and delays

rel_comp %>% 
  select(dv, task, retest_reliability, sample_size, days) %>%
  filter(days < 3600) %>%
  gather(key, value, -dv, -task) %>%
  ggplot(aes(value, fill=task))+
  geom_density(alpha=0.5, position='identity')+
  facet_wrap(~key, scales='free')+
  theme(legend.title = element_blank())

summary(rel_comp$sample_size[rel_comp$task == "survey"])
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
     11      38      73     126     153     791 
summary(rel_comp$sample_size[rel_comp$task == "task"])
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   10.0    30.0    47.0    78.6    81.0  1120.0 

The literature has smaller sized samples for task measures compared to survey measures that report retest reliability.

summary(lm(sample_size ~ task,rel_comp))

Call:
lm(formula = sample_size ~ task, data = rel_comp)

Residuals:
   Min     1Q Median     3Q    Max 
-115.5  -57.6  -33.6    7.4 1043.4 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)   126.50       8.47   14.94  < 2e-16 ***
tasktask      -47.89      10.79   -4.44 0.000011 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 127 on 581 degrees of freedom
Multiple R-squared:  0.0328,    Adjusted R-squared:  0.0311 
F-statistic: 19.7 on 1 and 581 DF,  p-value: 0.0000108

What predicts retest reliability in the literature? Task, sample size, days

mod1 = lmer(retest_reliability ~ task + (1|dv), rel_comp)
mod2 = lmer(retest_reliability ~ task + sample_size + (1|dv), rel_comp)

anova(mod1, mod2)
refitting model(s) with ML (instead of REML)
mod3 = lmer(retest_reliability ~ task + sample_size + days+ (1|dv), rel_comp)

anova(mod2, mod3)
refitting model(s) with ML (instead of REML)
summary(mod2)
Linear mixed model fit by REML ['lmerMod']
Formula: retest_reliability ~ task + sample_size + (1 | dv)
   Data: rel_comp

REML criterion at convergence: -377.2

Scaled residuals: 
   Min     1Q Median     3Q    Max 
-3.190 -0.488  0.127  0.550  2.575 

Random effects:
 Groups   Name        Variance Std.Dev.
 dv       (Intercept) 0.0174   0.132   
 Residual             0.0206   0.143   
Number of obs: 583, groups:  dv, 171

Fixed effects:
              Estimate Std. Error t value
(Intercept)  0.7235198  0.0213512    33.9
tasktask    -0.1405121  0.0258197    -5.4
sample_size -0.0001256  0.0000543    -2.3

Correlation of Fixed Effects:
            (Intr) tsktsk
tasktask    -0.780       
sample_size -0.319  0.117

Tasks have significantly lower reliability and reliability decreases with increasing sample size.

rel_comp %>% 
  ggplot(aes(sample_size, retest_reliability, color=task))+
  geom_smooth(method='lm')+
  geom_point(alpha = 0.2)+
  theme(legend.title = element_blank())+
  xlab("Sample Size")+
  ylab("Retest reliability")

I used to compare effect sizes of effect of task on reliability estimates in literature vs our results (just looking at the variables you find in the literature) but removed this analysis because
1. the estimates in the literature are not all the same statistic 2. from our data I was only using ICC’s 3. I was sampling from our bootstrapped results as many datapoints for each measure as there are in the literature but that didn’t seem like the best way to make use of our data to make it comparable to the literature.

Despite these problems the main result was that the effect size was much larger in our dataset compared to the literature but given the problems I think the sampling analyses below are more informative than this.

We also checked whether our results diverge most from studies with smaller sample sizes. Square difference between our mean estimate and the reliability from the literature decreases exponentially with sample size. The smaller the sample size in the literature the more the reliability estimate differs from our results. But this was a weak result because most of the studies in the literature have smaller sample sizes and you see both small and large deviations for these studies (these were not significant either).

Direct relation to our results

Correlation between our mean estimates from bootstrapped samples and the literature review for task variables

n_df = rel_comp %>% 
  group_by(dv) %>%
  tally()

lit_emp_cor = function(){
  
  boot_comp = data.frame()
  
  for(i in 1:length(unique(rel_comp$dv)) ){
    cur_dv = unique(rel_comp$dv)[i]
    n = n_df$n[n_df$dv == cur_dv]
    sample_df = boot_df %>% filter(dv == cur_dv)
    tmp = sample_n(sample_df, n)
    boot_comp = rbind(boot_comp, tmp)
  }  
  
  rm(cur_dv, n, sample_df, tmp)
  
  #check if cbind is ok
  # sum(boot_comp$dv == rel_comp$dv)
  #cbinding pearson because that is the most common metric in the lit
  rel_comp = cbind(rel_comp, boot_comp$pearson)
  #rename new column
  names(rel_comp)[which(names(rel_comp) == "boot_comp$pearson")] = "pearson"
  
  out = data.frame(task = NA, survey = NA)
  
  out$task = with(rel_comp %>% filter(task == "task"), cor(pearson, retest_reliability))
  
  out$survey = with(rel_comp %>% filter(task == "survey"), cor(pearson, retest_reliability))
  
  rel_comp = rel_comp[,-16]
  
  return(out)
}

lit_emp_cor_out = plyr::rdply(100, lit_emp_cor)

write.csv(lit_emp_cor_out,'/Users/zeynepenkavi/Dropbox/PoldrackLab/SRO_Retest_Analyses/output/tables/lit_emp_cor_out.csv')

summary(lit_emp_cor_out)
       .n             task           survey      
 Min.   :  1.0   Min.   :0.235   Min.   :0.0381  
 1st Qu.: 25.8   1st Qu.:0.263   1st Qu.:0.1162  
 Median : 50.5   Median :0.274   Median :0.1392  
 Mean   : 50.5   Mean   :0.274   Mean   :0.1381  
 3rd Qu.: 75.2   3rd Qu.:0.285   3rd Qu.:0.1582  
 Max.   :100.0   Max.   :0.323   Max.   :0.2478  

Noise ceiling

Model comparisons building model to predict the reliabilities in the literature from a sample from our results versus a sample form the literature.

sample one row per measure out of lit review r^2 of retest_reliability ~ sampled_reliability vs. r^2 of retest_reliability ~ mean_icc

coef of retest_reliability ~ sampled_reliability vs. coef of retest_reliability ~ mean_icc

comp_lit_pred <- function(df){
  
  sample_from_dv <- function(df){
    if(nrow(df)>1){
      row_num = sample(1:nrow(df),1)
      sample_row = df[row_num,]
      df = df[-row_num,]
      df$lit_predictor = sample_row$retest_reliability
    }
    return(df)
  }
  
  sampled_df = df %>%
    group_by(dv) %>%
    do(sample_from_dv(.)) 
  
  mod_lit = lm(retest_reliability ~ lit_predictor+scale(sample_size)+task, data=sampled_df)
  mod_boot = lm(retest_reliability ~ mean_pearson+scale(sample_size)+task, data=sampled_df)
  
  out = data.frame(r2_lit = summary(mod_lit)$r.squared,
                   r2_boot = summary(mod_boot)$r.squared,
                   m_lit = coef(summary(mod_lit))["lit_predictor","Estimate"],
                   m_boot = coef(summary(mod_boot))["mean_pearson","Estimate"])
  
  return(out)
}

comp_lit_pred_out = plyr::rdply(1000, comp_lit_pred(rel_comp))
tmp = comp_lit_pred_out %>%
  select(-.n) %>%
  gather(key, value) %>%
  separate(key, c("stat", "sample"), sep = "_")
 
tmp$stat = as.factor(tmp$stat)
levels(tmp$stat) <- c("coefficient", expression(r^"2"))


tmp %>%  
  ggplot(aes(value, fill=sample))+
  geom_density(alpha = 0.5, position='identity', color=NA)+
  facet_grid(.~stat, scales='free', labeller = label_parsed)+
  scale_fill_discrete(breaks=c("boot","lit"),
                       labels=c("Empirical", "Literature"),
                      name="Prediction")+
  xlab('')+
  ylab('')

ggsave('Lit_Noise_Ceiling.jpg', device = "jpeg", path = "/Users/zeynepenkavi/Dropbox/PoldrackLab/SRO_Retest_Analyses/output/figures/", width = 10, height = 4, units = "in", limitsize = FALSE, dpi = 100)
with(comp_lit_pred_out, t.test(r2_lit, r2_boot, paired=T))

    Paired t-test

data:  r2_lit and r2_boot
t = 29, df = 1000, p-value <2e-16
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 0.02467 0.02821
sample estimates:
mean of the differences 
                0.02644 
with(comp_lit_pred_out, t.test(m_lit, m_boot))

    Welch Two Sample t-test

data:  m_lit and m_boot
t = 18, df = 1300, p-value <2e-16
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 0.02791 0.03472
sample estimates:
mean of x mean of y 
   0.3334    0.3020 

Relationship between reliability metrics (point estimates)

Based on Weir (2005) ICC(3,k) does not take in to account within subject differences between two time points (i.e. the fixed effect of time/systematic error). Thus, it is well approximated by Pearson’s r and subject to similar criticisms. Weir (2005) suggests reporting at least this systematic error effect size if one chooses to report with ICC(3,k). Based on his conclusions here I report:
- ICC(3,k): As Dave clarified this ranges from 1 to -1/(number of repeated measures -1) so in our case this range would be [-1, 1]; larger values would mean that the two scores of a subject for a given measure are more similar to each other than they are to scores of other people
- “ICC is reflective of the ability of a test to differentiate between different individuals” - partial \(\eta^2\) for time (\(SS_{time}/SS_{within}\)): effect size of time
- SEM (\(\sqrt(MS_{error})\)): standard error of measurement; the smaller the better. It “quantifies the precision of individual scores on a test” and is not dependent on the sample in the way ICC is since it doesn’t depend on between subject reliability (at least in this formulation) but is unit-dependent.

We calculated 74 measures for surveys and 372 measures for cognitive tasks.

Though we are primarily reporting ICC’s as our metric of reliability the results don’t change depending on the metric chosen. Here we plot point estimates of three different reliability metrics against each other (ICC, Pearson, Spearman). The bootstrapped version is essentially the same but the plots are busier due to more datapoints.

table(rel_df$task)

survey   task 
    74    372 
p1 = rel_df %>%
  ggplot(aes(spearman, icc, col=task))+
  geom_point()+
  theme_bw()+
  theme(legend.title = element_blank(),
        legend.position = "none")+
  geom_abline(intercept = 0, slope=1)

p2 = rel_df %>%
  ggplot(aes(pearson, icc, col=task))+
  geom_point()+
  theme_bw()+
  theme(legend.title = element_blank(),
        legend.position = "none")+
  geom_abline(intercept = 0, slope=1)

p3 = rel_df %>%
  ggplot(aes(pearson, spearman, col=task))+
  geom_point()+
  theme_bw()+
  theme(legend.title = element_blank(),
        legend.position = "none")+
  geom_abline(intercept = 0, slope=1)

grid.arrange(p1, p2, p3, nrow=1)

ggsave('Metric_Scatterplots.jpg', plot = grid.arrange(p1, p2, p3, nrow=1), device = "jpeg", path = "/Users/zeynepenkavi/Dropbox/PoldrackLab/SRO_Retest_Analyses/output/figures/", width = 12, height = 4, units = "in", limitsize = FALSE, dpi = 100)

As the scatter plots depict the correlations between different types reliability metrics were very high.

cor(rel_df[,c('spearman', 'icc', 'pearson')])
         spearman    icc pearson
spearman   1.0000 0.9324  0.9577
icc        0.9324 1.0000  0.9800
pearson    0.9577 0.9800  1.0000

Note: Some variables have <0 ICC’s. This would be the case if the \(MS_{error}\)>\(MS_{between}\). Data for these variables have no relationship between the two time points.

Summary of all measure reliabilities

Summarized bootstrapped reliabilities

boot_df %>%
  group_by(dv) %>%
  summarise(icc_median = quantile(icc, probs = 0.5),
            icc_2.5 = quantile(icc, probs = 0.025),
            icc_97.5 = quantile(icc, probs = 0.975),
            spearman_median = quantile(spearman, probs = 0.5),
            spearman_2.5 = quantile(spearman, probs = 0.025),
            spearman_97.5 = quantile(spearman, probs = 0.975)) %>%
  datatable() %>%
  formatRound(columns=c('icc_median', 'icc_2.5', 'icc_97.5', 'spearman_median', 'spearman_2.5', 'spearman_97.5'), digits=3)
# Df wrangling for plotting
tmp = measure_labels %>%
  mutate(dv = as.character(dv)) %>%
  left_join(boot_df[,c("dv", "icc", "spearman")], by = 'dv') 

tmp = tmp %>%
  separate(dv, c("task_group", "var"), sep="\\.",remove=FALSE,extra="merge") %>%
  mutate(task_group = factor(task_group, levels = task_group[order(task)])) %>%
  separate(var, c("var"), sep="\\.",remove=TRUE,extra="drop") %>%
  mutate(task_group = gsub("_", " ", task_group),
         var = gsub("_", " ", var)) %>%
  arrange(task_group, var)

tmp = tmp %>%
  left_join(rel_df[,c("dv", "icc")], by = "dv") %>%
  rename(icc = icc.x, point_est = icc.y)

#Manual correction
tmp = tmp %>%
  mutate(task = ifelse(task_group == 'holt laury survey', "task", as.character(task))) %>%
  mutate(task_group = ifelse(task_group == "psychological refractory period two choices", "psychological refractory period", ifelse(task_group == "angling risk task always sunny", "angling risk task",task_group))) %>%
  mutate(task_group = gsub("survey", "", task_group))

Variable level summary of bootstrapped reliabilities.

p4 <- tmp %>%
  filter(task == 'task',
         raw_fit == 'raw') %>%
ggplot(aes(y = var, x = icc)) + 
  geom_point(color = '#00BFC4')+
  geom_point(aes(y = var, x = point_est), color = "black")+
  facet_grid(task_group~., switch = "y", scales = "free_y", space = "free_y", labeller = label_wrap_gen(width=20)) +
  theme(panel.spacing = unit(0.75, "lines"), 
        strip.placement = "outside",
        strip.text.y = element_text(angle=180, size=36),
        axis.text.y = element_text(size=20),
        panel.background = element_rect(fill = NA),
        panel.grid.major = element_line(colour = "grey80")) + 
  xlab("")+
  ylab("")+
  scale_x_continuous(limits = c(-0.25,1), breaks=c(-0.25, 0, 0.25, 0.5, 0.75, 1))+
  scale_y_discrete(labels = function(x) str_wrap(x, width = 10))+
  geom_vline(xintercept = 0, color = "red", size = 1)

p5 <- tmp %>%
  filter(task == 'survey') %>%
ggplot(aes(y = var, x = icc)) + 
  geom_point(color = '#F8766D')+
  geom_point(aes(y = var, x = point_est), color = "black")+
  facet_grid(task_group~., switch = "y", scales = "free_y", space = "free_y", labeller = label_wrap_gen(width=20)) +
  theme(panel.spacing = unit(0.75, "lines"), 
        strip.placement = "outside",
        strip.text.y = element_text(angle=180, size=36),
        axis.text.y = element_text(size=20),
        panel.background = element_rect(fill = NA),
        panel.grid.major = element_line(colour = "grey80")) + 
  xlab("")+
  ylab("")+
  scale_x_continuous(limits = c(-0.25,1), breaks=c(-0.25, 0, 0.25, 0.5, 0.75, 1))+
  scale_y_discrete(labels = function(x) str_wrap(x, width = 10))+
  geom_vline(xintercept = 0, color = "red", size = 1)
  
p6 <- arrangeGrob(p4, p5,nrow=1)

ggsave('Bootstrap_Raw_Var_Plot.jpg', plot = p6, device = "jpeg", path = "/Users/zeynepenkavi/Dropbox/PoldrackLab/SRO_Retest_Analyses/output/figures/", width = 36, height = 72, units = "in", limitsize = FALSE, dpi=50)

rm(p4, p5, p6)

p4_t <- tmp %>%
  filter(task == 'task') %>%
ggplot(aes(x = factor(task_group, levels=rev(unique(task_group))), y = icc)) + 
  geom_violin(fill='#00BFC4')+
  theme_bw() + 
  theme(axis.text.y = element_text(size=30))+
  xlab("")+
  ylab("")+
  scale_y_continuous(limits = c(-0.25,1), breaks=c(-0.25, 0, 0.25, 0.5, 0.75, 1))+
  scale_x_discrete(labels = function(x) str_wrap(x, width = 15))+
  geom_hline(yintercept = 0, color = "red", size = 1)+
  coord_flip()

p5_t <- tmp %>%
  filter(task == 'survey') %>%
ggplot(aes(x = factor(task_group, levels=rev(unique(task_group))), y = icc)) + 
  geom_violin(fill='#F8766D')+
  theme_bw() + 
  theme(axis.text.y = element_text(size=43))+
  xlab("")+
  ylab("")+
  scale_y_continuous(limits = c(-0.25,1), breaks=c(-0.25, 0, 0.25, 0.5, 0.75, 1))+
  scale_x_discrete(labels = function(x) str_wrap(x, width = 10))+
  geom_hline(yintercept = 0, color = "red", size = 1)+
  coord_flip()
  
p6_t <- arrangeGrob(p4_t, p5_t,nrow=1)

ggsave('Bootstrap_Poster_Plot_t.jpg', plot = p6_t, device = "jpeg", path = "../output/figures/", width = 27, height = 48, units = "in", limitsize = FALSE, dpi = 100)

Example of the overlaying procedure.

p1<- tmp %>%
  filter(grepl('selection_optimization', dv)) %>%
ggplot(aes(x = var, y = icc)) + 
  geom_violin(fill='#F8766D')+
  theme_bw() + 
  theme(axis.text.y = element_text(size=30))+
  xlab("")+
  ylab("")+
  scale_y_continuous(limits = c(-0.25,1), breaks=c(-0.25, 0, 0.25, 0.5, 0.75, 1))+
  scale_x_discrete(labels = function(x) str_wrap(x, width = 15))+
  geom_hline(yintercept = 0, color = "red", size = 1)+
  coord_flip()

p2<- tmp %>%
  filter(grepl('selection_optimization', dv)) %>%
ggplot(aes(x = factor(task_group, levels=rev(unique(task_group))), y = icc, group=dv)) + 
  geom_violin(fill='#F8766D', position = 'identity')+
  theme_bw() + 
  theme(axis.text.y = element_text(size=30))+
  xlab("")+
  ylab("")+
  scale_y_continuous(limits = c(-0.25,1), breaks=c(-0.25, 0, 0.25, 0.5, 0.75, 1))+
  scale_x_discrete(labels = function(x) str_wrap(x, width = 15))+
  geom_hline(yintercept = 0, color = "red", size = 1)+
  coord_flip()

p3<- tmp %>%
  filter(grepl('selection_optimization', dv)) %>%
ggplot(aes(x = factor(task_group, levels=rev(unique(task_group))), y = icc)) + 
  geom_violin(fill='#F8766D', position = 'identity')+
  theme_bw() + 
  theme(axis.text.y = element_text(size=30))+
  xlab("")+
  ylab("")+
  scale_y_continuous(limits = c(-0.25,1), breaks=c(-0.25, 0, 0.25, 0.5, 0.75, 1))+
  scale_x_discrete(labels = function(x) str_wrap(x, width = 15))+
  geom_hline(yintercept = 0, color = "red", size = 1)+
  coord_flip()

p4 <- arrangeGrob(p1,p2, p3,nrow=1)

ggsave('Bootstrap_Example_Plot_t.jpg', plot = p4, device = "jpeg", path = "../output/figures/", width = 41, height = 5, units = "in", limitsize = FALSE, dpi = 100)

rm(p1, p2, p3, p4)

Survey vs Tasks

Comparison of survey measures to cognitive task measures in the bootstrapped results. Multilevel model with random intercepts for each measure and fixed effect of survey versus cognitive measure.

boot_df = boot_df %>%
    mutate(task = ifelse(grepl("survey",dv), "survey","task"),
           task = ifelse(grepl("holt",dv), "task", task))

boot_df %>%
  group_by(task) %>%
  summarise(icc_median = quantile(icc, probs = 0.5),
            icc_2.5 = quantile(icc, probs = 0.025),
            icc_97.5 = quantile(icc, probs = 0.975),
            spearman_median = quantile(spearman, probs = 0.5),
            spearman_2.5 = quantile(spearman, probs = 0.025),
            spearman_97.5 = quantile(spearman, probs = 0.975),
            num_vars = n()/1000) %>%
  datatable() %>%
  formatRound(columns=c('icc_median', 'icc_2.5', 'icc_97.5', 'spearman_median', 'spearman_2.5', 'spearman_97.5'), digits=3)
summary(lmerTest::lmer(icc ~  task + (1|dv), boot_df))
Linear mixed model fit by REML t-tests use Satterthwaite approximations
  to degrees of freedom [lmerMod]
Formula: icc ~ task + (1 | dv)
   Data: boot_df

REML criterion at convergence: -942508

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-13.201  -0.401   0.024   0.459   7.051 

Random effects:
 Groups   Name        Variance Std.Dev.
 dv       (Intercept) 0.05354  0.2314  
 Residual             0.00701  0.0837  
Number of obs: 446000, groups:  dv, 446

Fixed effects:
            Estimate Std. Error       df t value Pr(>|t|)    
(Intercept)   0.8725     0.0269 449.0000    32.4   <2e-16 ***
tasktask     -0.3903     0.0295 449.0000   -13.2   <2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Correlation of Fixed Effects:
         (Intr)
tasktask -0.913

Variance breakdown

The quantiative explanation for the difference in reliability estimates between surveys and tasks, as recently detailed by Hedge et al. (2017), lies in the difference in sources of variance between these measures. Specifically, the ICC is calculated as the ratio of variance between subjects variance to all sources of variance. Thus, measures with high between subjects variance would have high test-retest reliability. Intuitively, measures with high between subjects variance are also better suited for individual difference analyses as they would capture the differences between the subjects in a sample.

Here we first plot the percentage of variance explained by the three sources of variance for the point estimates of measure reliabilities. The plot only includes raw measures (no DDM parameters) and the measures are ranked by percentage of between subject variability for each task/survey (i.e. the best to worst individual difference measure for each task/survey). Then we compare statistically whether the percentage of variance explained by these sources differ between tasks and surveys.

tmp = rel_df %>%
  mutate(var_subs_pct = var_subs/(var_subs+var_ind+var_resid)*100,
         var_ind_pct = var_ind/(var_subs+var_ind+var_resid)*100, 
         var_resid_pct = var_resid/(var_subs+var_ind+var_resid)*100) %>%
  select(dv, task, var_subs_pct, var_ind_pct, var_resid_pct) %>%
  mutate(dv = factor(dv, levels = dv[order(task)])) %>%
  separate(dv, c("task_group", "var"), sep="\\.",remove=FALSE,extra="merge") %>%
  mutate(task_group = factor(task_group, levels = task_group[order(task)])) %>%
  arrange(task_group, var_subs_pct) %>%
  mutate(rank = row_number()) %>%
  arrange(task, task_group, rank) %>%
  gather(key, value, -dv, -task_group, -var, -task, -rank) %>%
  ungroup()%>%
  mutate(task_group = gsub("_", " ", task_group),
         var = gsub("_", " ", var)) %>%
  mutate(task_group = ifelse(task_group == "psychological refractory period two choices", "psychological refractory period", ifelse(task_group == "angling risk task always sunny", "angling risk task",task_group))) %>%
  mutate(task_group = gsub("survey", "", task_group)) %>%
  filter(task=="task",
         !grepl("EZ|hddm", dv))%>%
  arrange(task_group, rank)
labels = tmp %>%
  distinct(dv, .keep_all=T)

p1 <- tmp %>%
  ggplot(aes(x=factor(rank), y=value, fill=factor(key, levels = c("var_resid_pct", "var_ind_pct", "var_subs_pct"))))+
  geom_bar(stat='identity', alpha = 0.75, color='#00BFC4')+
  scale_x_discrete(breaks = labels$rank,
                       labels = labels$var)+
  coord_flip()+
  facet_grid(task_group~., switch = "y", scales = "free_y", space = "free_y") +
  theme(panel.spacing = unit(0.5, "lines"), 
        strip.placement = "outside",
        strip.text.y = element_text(angle=180),
        panel.background = element_rect(fill = NA),
        panel.grid.major = element_line(colour = "grey85"),
        legend.position = 'bottom')+
  theme(legend.title = element_blank())+
  scale_fill_manual(breaks = c("var_subs_pct", "var_ind_pct", "var_resid_pct"),
                      labels = c("Variance between individuals",
                                 "Variance between sessions",
                                 "Error variance"),
                  values=c("grey65", "grey45", "grey25"))+
  ylab("")+
  xlab("")

tmp = rel_df %>%
  mutate(var_subs_pct = var_subs/(var_subs+var_ind+var_resid)*100,
         var_ind_pct = var_ind/(var_subs+var_ind+var_resid)*100, 
         var_resid_pct = var_resid/(var_subs+var_ind+var_resid)*100) %>%
  select(dv, task, var_subs_pct, var_ind_pct, var_resid_pct) %>%
  mutate(dv = factor(dv, levels = dv[order(task)])) %>%
  separate(dv, c("task_group", "var"), sep="\\.",remove=FALSE,extra="merge") %>%
  mutate(task_group = factor(task_group, levels = task_group[order(task)])) %>%
  arrange(task_group, var_subs_pct) %>%
  mutate(rank = row_number()) %>%
  arrange(task, task_group, rank) %>%
  gather(key, value, -dv, -task_group, -var, -task, -rank) %>%
  ungroup()%>%
  mutate(task_group = gsub("_", " ", task_group),
         var = gsub("_", " ", var)) %>%
  mutate(task_group = ifelse(task_group == "psychological refractory period two choices", "psychological refractory period", ifelse(task_group == "angling risk task always sunny", "angling risk task",task_group))) %>%
  mutate(task_group = gsub("survey", "", task_group)) %>%
  filter(task=="survey")%>%
  arrange(task_group, rank)
labels = tmp %>%
  distinct(dv, .keep_all=T)

p2 <- tmp %>%
  ggplot(aes(x=factor(rank), y=value, fill=factor(key, levels = c("var_resid_pct", "var_ind_pct", "var_subs_pct"))))+
  geom_bar(stat='identity', alpha = 0.75)+
  geom_bar(stat='identity', color='#F8766D', show.legend=FALSE)+
  scale_x_discrete(breaks = labels$rank,
                       labels = labels$var)+
  coord_flip()+
  facet_grid(task_group~., switch = "y", scales = "free_y", space = "free_y") +
  theme(panel.spacing = unit(0.5, "lines"), 
        strip.placement = "outside",
        strip.text.y = element_text(angle=180),
        panel.background = element_rect(fill = NA),
        panel.grid.major = element_line(colour = "grey85"),
        legend.position = 'bottom')+
  theme(legend.title = element_blank())+
  scale_fill_manual(breaks = c("var_subs_pct", "var_ind_pct", "var_resid_pct"),
                      labels = c("Variance between individuals",
                                 "Variance between sessions",
                                 "Error variance"),
                  values=c("grey65", "grey45", "grey25"))+
  ylab("")+
  xlab("")

mylegend<-g_legend(p2)

p3 <- arrangeGrob(arrangeGrob(p1 +theme(legend.position="none"),
                         p2 + theme(legend.position="none"),
                         nrow=1),
             mylegend, nrow=2,heights=c(10, 1))

ggsave('Variance_Breakdown_Plot.jpg', plot = p3, device = "jpeg", path = "/Users/zeynepenkavi/Dropbox/PoldrackLab/SRO_Retest_Analyses/output/figures", width = 24, height = 20, units = "in", dpi = 100)
rm(tmp, labels, p1, p2 , p3)

Comparing types of variance for survey vs task measures: Survey measures have higher between subject variability

Note: This analysis includes DDM variables too.

Running separate models for different sources of variance because interactive model with variance type*task seemed too complicated.

First we find that task measures have a smaller percentage of their overall variance explained by variability between subjects compared to survey measures.

tmp = boot_df %>%
  mutate(var_subs_pct = var_subs/(var_subs+var_ind+var_resid)*100,
         var_ind_pct = var_ind/(var_subs+var_ind+var_resid)*100, 
         var_resid_pct = var_resid/(var_subs+var_ind+var_resid)*100) %>%
  select(dv, task, var_subs_pct, var_ind_pct, var_resid_pct) 

summary(lmerTest::lmer(var_subs_pct~task+(1|dv),tmp%>%select(-var_ind_pct,-var_resid_pct)))
Linear mixed model fit by REML t-tests use Satterthwaite approximations
  to degrees of freedom [lmerMod]
Formula: var_subs_pct ~ task + (1 | dv)
   Data: tmp %>% select(-var_ind_pct, -var_resid_pct)

REML criterion at convergence: 3304624

Scaled residuals: 
   Min     1Q Median     3Q    Max 
-4.316 -0.621  0.103  0.647  5.381 

Random effects:
 Groups   Name        Variance Std.Dev.
 dv       (Intercept) 188      13.7    
 Residual              96       9.8    
Number of obs: 446000, groups:  dv, 446

Fixed effects:
            Estimate Std. Error     df t value Pr(>|t|)    
(Intercept)    80.33       1.60 443.00    50.4   <2e-16 ***
tasktask      -28.81       1.75 443.00   -16.5   <2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Correlation of Fixed Effects:
         (Intr)
tasktask -0.913

We also find that a significantly larger percentage of their variance is explained by between session variability. Larger between session variability suggests systematic differences between the two sessions. Such systematic effects can be due to e.g. learning effects as explored later.

summary(lmerTest::lmer(var_ind_pct~task+(1|dv),tmp%>%select(-var_subs_pct,-var_resid_pct)))
Linear mixed model fit by REML t-tests use Satterthwaite approximations
  to degrees of freedom [lmerMod]
Formula: var_ind_pct ~ task + (1 | dv)
   Data: tmp %>% select(-var_subs_pct, -var_resid_pct)

REML criterion at convergence: 3611691

Scaled residuals: 
   Min     1Q Median     3Q    Max 
-4.412 -0.651 -0.139  0.584  4.736 

Random effects:
 Groups   Name        Variance Std.Dev.
 dv       (Intercept) 239      15.5    
 Residual             191      13.8    
Number of obs: 446000, groups:  dv, 446

Fixed effects:
            Estimate Std. Error     df t value Pr(>|t|)    
(Intercept)     9.84       1.80 444.00    5.47  7.4e-08 ***
tasktask       14.36       1.97 444.00    7.29  1.4e-12 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Correlation of Fixed Effects:
         (Intr)
tasktask -0.913
summary(lmerTest::lmer(var_resid_pct~task+(1|dv),tmp%>%select(-var_subs_pct,-var_ind_pct)))
Linear mixed model fit by REML t-tests use Satterthwaite approximations
  to degrees of freedom [lmerMod]
Formula: var_resid_pct ~ task + (1 | dv)
   Data: tmp %>% select(-var_subs_pct, -var_ind_pct)

REML criterion at convergence: 2777592

Scaled residuals: 
   Min     1Q Median     3Q    Max 
-6.323 -0.475  0.018  0.530  6.343 

Random effects:
 Groups   Name        Variance Std.Dev.
 dv       (Intercept) 75.2     8.67    
 Residual             29.4     5.43    
Number of obs: 446000, groups:  dv, 446

Fixed effects:
            Estimate Std. Error     df t value Pr(>|t|)    
(Intercept)     9.83       1.01 446.00    9.75   <2e-16 ***
tasktask       14.45       1.10 446.00   13.09   <2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Correlation of Fixed Effects:
         (Intr)
tasktask -0.913
tmp_save = tmp%>%
  gather(key, value, -dv, -task) %>%
  group_by(task, key) %>%
  summarise(median = median(value),
            sd = sd(value)) %>%
  mutate(key = ifelse(key == 'var_ind_pct', 'Between session variance', ifelse(key == 'var_subs_pct', 'Between subjects variance', ifelse(key == 'var_resid_pct', 'Residual variance',NA)))) %>%
  rename(Median = median, SD = sd) %>%
  arrange(task, key)

tmp_save
sjt.df(tmp_save %>% mutate_if(is.numeric, funs(round(., 3))), describe=F, hide.progress = TRUE, show.rownames = FALSE, file = "/Users/zeynepenkavi/Dropbox/PoldrackLab/SRO_Retest_Analyses/output/tables/var_breakdown_summary.doc")
task key Median SD
survey Between session variance 5.384 11.353
survey Between subjects variance 83.355 11.686
survey Residual variance 8.976 4.325
task Between session variance 18.077 22.114
task Between subjects variance 52.548 17.681
task Residual variance 22.818 11.012

Summarizing for clearer presentation. This graph is currently using the bootstrapped reliabilities and is therefore messier than if just using the point estimates.

tmp %>%
  gather(key, value, -dv, -task) %>%
  group_by(task, key) %>%
  summarise(mean_pct = mean(value),
            sd_pct = sd(value),
            n = n()) %>%
  mutate(cvl = qt(0.025, n-1),
         cvu = qt(0.975, n-1),
         cil = mean_pct+(sd_pct*cvl)/sqrt(n),
         ciu = mean_pct+(sd_pct*cvu)/sqrt(n),
         sem_pct = sd_pct/sqrt(n)) %>%
  ggplot(aes(factor(key, levels = c("var_subs_pct", "var_ind_pct", "var_resid_pct"),
                    labels = c("Variance between individuals",
                               "Variance between sessions",
                               "Error variance")), mean_pct, color=task))+
  geom_point(position=position_dodge(width = 0.25), size = 4)+
  # geom_errorbar(aes(ymin=mean_pct-sd_pct, ymax=mean_pct+sd_pct), position=position_dodge(width = 0.25), width=0, size=2)+
  geom_errorbar(aes(ymin=cil, ymax=ciu), position=position_dodge(width = 0.25), width=0.25, size=2)+
  theme_bw()+
  xlab('')+
  ylab('Percent')+
  theme(legend.title = element_blank(),
        legend.text = element_text(size=14),
        legend.position = 'bottom',
        axis.text = element_text(size=12),
        axis.title.y = element_text(size=12))+
  scale_x_discrete(labels = function(x) str_wrap(x, width = 15))

ggsave('Variance_Breakdown_DotPlot.jpg', device = "jpeg", path = "/Users/zeynepenkavi/Dropbox/PoldrackLab/SRO_Retest_Analyses/output/figures", width = 6, height = 5, units = "in", dpi = 100)

Systematic effects between time points

Anova time effects

The type of ICC we have chosen does not take within subject (between session/systematic) variance in to account. This is why Weir recommends checking whether there is a significant change based on time and examining the SEMs. These systematic effects could be meaningful and important to account for for some measures (e.g. task measures that show learning effects).

Had we chosen another kind of ICC taking this source of variance into account (e.g. 2,1 or 2,k) they could have suggested that tasks have lower reliability.

Doing a simple t-test on the difference score alone would not be a very rigourous way of testing whether any change is meaningful because two distributions from both time points with error would be compared to each other. Fortunately there are ways to take the error for both measurements in to account.

To check whether a measure shows systematic differences between the two time points in a meaningful number of bootstrapped samples we can: check if the effect of time is significant in each bootstrapped sample and filter variables that have more than 5% of the boostrapped samples showing significant time effects.

Another way might be to compute confidence intervals using SEMs as described in the second half of Weir (2005) and check what percent of participants have scores that fall out of this range. I haven’t pursued this for now.

Here we ask: Which variables have significant time effects in more than 5% of the bootstrapped samples?

23/74 survey measures 133/372 task measures

boot_df %>%
  select(dv, p_time, task) %>%
  mutate(time_effect_sig = ifelse(p_time<0.05,1,0)) %>%
  group_by(dv)%>%
  summarise(pct_sig_time_effect = sum(time_effect_sig)/10,
            task = unique(task))%>%
  filter(pct_sig_time_effect>5) %>%
  arrange(task,-pct_sig_time_effect) %>%
  ungroup()%>%
  group_by(task) %>%
  summarise(count=n())

Are these significantly different between surveys and tasks? No.

chisq.test(matrix(data = c(23,74-23,133, 372-133), nrow=2))

    Pearson's Chi-squared test with Yates' continuity correction

data:  matrix(data = c(23, 74 - 23, 133, 372 - 133), nrow = 2)
X-squared = 0.4, df = 1, p-value = 0.5

Are they predominantly in one or the other direction and does that differ depending on survey vs task?

boot_df %>%
  select(dv, p_time, task, pearson) %>%
  mutate(time_effect_sig = ifelse(p_time<0.05,1,0)) %>%
  group_by(dv)%>%
  summarise(pct_sig_time_effect = sum(time_effect_sig)/10,
            task = unique(task),
            mean_pearson = mean(pearson))%>%
  filter(pct_sig_time_effect>5) %>%
  arrange(task,-pct_sig_time_effect) %>%
  arrange(mean_pearson) 

SEM CI calculation

Step 1: T = Grand mean + ICC * (Subject score - Grand mean) Step 2: SEP = SD of both measurements * sqrt(1-ICC^2)

Here I calculate the proportion of subjects that move more than one standard error of prediction (SEP) in t2 compared to t1 for each measure.

This is odd to think about because the larger the ICC of a measure the smaller the SEP. So very small differences between the two time points can be categorized as ‘meaningful’ based on the tiny SEP.

What you are interested in is not necessarily whether individuals change at all between the two time points though. You want to know if this change is systematic in one direction.

Here I calculate the proportion of people showing ‘meaningful’ changes in one direction or the other. To integrate both of these direction I subtracted one from the other and filtered the variables that have more than 5% of the participants showing meaningful change in one direction over the other (so if a variable has 10 participants showing difference in one and 10 in the other direction this would cancel out but a variables with 15 people showing a positive and 5 negative change would remain).

get_ind_ci = function(dv_var){
  matched = match_t1_t2(dv_var)
  grand_mean = mean(matched$score)
  grand_sd = sd(matched$score)
  dv_icc = get_icc(dv_var)
  sep = grand_sd * sqrt(1-(dv_icc^2))
  matched = matched %>% 
    spread(time, score) %>%
    rename("t1"="1", "t2"="2") %>%
    mutate(true_score = grand_mean+(dv_icc*(t1-grand_mean)),
           ci_up = true_score+(1.96*sep),
           ci_low = true_score-(1.96*sep),
           t2_above_ci = ifelse(t2>ci_up,1,0),
           t2_below_ci = ifelse(t2<ci_low,1, 0))
    return(matched)
}

get_prop_out_ci = function(dv_var){
  get_ind_ci(dv_var) %>%
    summarise(prop_above_ci = sum(t2_above_ci)/n(),
              prop_below_ci = sum(t2_below_ci/n()))
}

#Create df of point estimate reliabilities
ind_ci_df <- data.frame(prop_above_ci = rep(NA, length(numeric_cols)),
                        prop_below_ci = rep(NA, length(numeric_cols)))

row.names(ind_ci_df) <- numeric_cols

for(i in 1:length(numeric_cols)){
    ind_ci_df[numeric_cols[i], 'prop_above_ci'] <- get_prop_out_ci(numeric_cols[i])$prop_above_ci 
    ind_ci_df[numeric_cols[i], 'prop_below_ci'] <- get_prop_out_ci(numeric_cols[i])$prop_below_ci 
}

ind_ci_df$dv = row.names(ind_ci_df)
row.names(ind_ci_df) = seq(1:nrow(ind_ci_df))
ind_ci_df$task = 'task'
ind_ci_df[grep('survey', ind_ci_df$dv), 'task'] = 'survey'
ind_ci_df[grep('holt', ind_ci_df$dv), 'task'] = "task"
ind_ci_df = ind_ci_df %>%
  select(dv, task, prop_above_ci, prop_below_ci)

mean_diff_df = ind_ci_df %>%
  mutate(prop_one_direction = prop_above_ci-prop_below_ci) %>%
  filter(abs(prop_one_direction)>0.05) %>%
  arrange(-abs(prop_one_direction))

mean_diff_df
# write.csv(mean_diff_df, '/Users/zeynepenkavi/Dropbox/PoldrackLab/SRO_Retest_Analyses/output/tables/mean_diff_df.csv')

This doesn’t make it easier to interpret whether it is a performance improvement or decline. In fact ‘performance’ isn’t even necessarily the correct term here. Using the valence assignments of the measures look at whether it translates to an increase or decrease in self-regulation. Of the 66 this makes sense for 54 variables.

mean_diff_df = read.csv('/Users/zeynepenkavi/Dropbox/PoldrackLab/SRO_Retest_Analyses/output/tables/mean_diff_df.csv')

table(as.character(mean_diff_df$valence), useNA = "ifany")

 Neg  Pos <NA> 
  24   30   12 
mean_diff_df %>%
  mutate(sc_increase = ifelse(prop_one_direction>0&valence=="Pos",1,ifelse(prop_one_direction<0&valence=="Neg",1,0)),
         sc_decrease = ifelse(prop_one_direction>0&valence=="Neg",1,ifelse(prop_one_direction<0&valence=="Pos",1,0))) %>%
summarise(sum_sc_increase = sum(sc_increase,na.rm=T),
          sum_sc_decrease = sum(sc_decrease,na.rm=T))

Variables that show more than 5% difference in a direction do not depend on whether they translate to a self control increase or decrease.

chisq.test(matrix(data = c(23,54-23,31,54-31), nrow=2))

    Pearson's Chi-squared test with Yates' continuity correction

data:  matrix(data = c(23, 54 - 23, 31, 54 - 31), nrow = 2)
X-squared = 1.8, df = 1, p-value = 0.2

Do they differ depending on whether they are task or survey variables? No.

chisq.test(matrix(data = c(5,74-5,61,372-61), nrow=2))

    Pearson's Chi-squared test with Yates' continuity correction

data:  matrix(data = c(5, 74 - 5, 61, 372 - 61), nrow = 2)
X-squared = 3.8, df = 1, p-value = 0.05
rm(mean_diff_df)

Task Reliabilities

Here we summarize the results on a task level to make it more digestable and easier to make contact with the literature.

We reduce the list of task measures to a list of one per task by averaging only the raw measures from all the trials in a task. We chose to reduce the information in this manner to avoid any bias stemming from differential amount of interest and procedures applied to certain tasks over others (e.g. a task can have over 10 measures because it has multiple conditions and we have chosen to fit DDM’s for specific conditions while another might only have 2 due to our relative inexperience and lack of interest in it). We check whether the number of trials in a task has a significant effect on these average reliabilities of raw measures as well.

We filter out the DDM parameters and measures for specific contrasts. Note that this does leave some tasks with measures that are model fits and/or for specific conditions (because at least the current datasets do not include measures that are based on all the trials and are raw though I could dive in to variables_exhaustive for such measures. For example the average relialibility for Kirby is based on three discount rates for specific conditions.). Here’s the order of tasks by mean reliability sorted for ICC and then Spearman’s \(\rho\).

tmp = measure_labels %>%
  mutate(dv = as.character(dv)) %>%
  filter(task == 'task') %>%
  left_join(boot_df[,c("dv", "icc", "spearman")], by = 'dv') %>%
  filter(overall_difference != 'difference' & raw_fit %in% c('EZ', 'hddm') == FALSE)%>%
  separate(dv, c('task_name', 'extra_1', 'extra_2'), sep = '\\.',remove=FALSE) %>%
  select(-extra_1, -extra_2) %>%
  group_by(task_name) %>%
  summarise(median_icc = median(icc),
            median_spearman = median(spearman),
            icc_2.5 = quantile(icc, probs = 0.025),
            icc_97.5 = quantile(icc, probs = 0.975),
            min_spearman = min(spearman),
            max_spearman = max(spearman),
            num_measures = n()/1000,
            mean_num_trials = round(mean(num_all_trials)))%>%
  arrange(-median_icc, -median_spearman)

tmp %>%
  datatable() %>%
  formatRound(columns=c('median_spearman', 'median_icc',
                        'min_spearman', 'icc_2.5',
                        'max_spearman', 'icc_97.5'), digits=3)
tmp = tmp%>%
  select(-median_spearman, -min_spearman, -max_spearman) %>%
  mutate(task_name = gsub("_", " ", task_name),
         task_name = gsub("(^|[[:space:]])([[:alpha:]])", "\\1\\U\\2", task_name, perl=TRUE))

names(tmp) = gsub("_", " ", names(tmp))
names(tmp) = gsub("(^|[[:space:]])([[:alpha:]])", "\\1\\U\\2", names(tmp), perl=TRUE)

sjt.df(tmp %>% mutate_if(is.numeric, funs(round(., 3))), describe=F, hide.progress = TRUE, show.rownames = FALSE, file = "/Users/zeynepenkavi/Dropbox/PoldrackLab/SRO_Retest_Analyses/output/tables/task_rel_table.doc")
Task Name Median Icc Icc 2.5 Icc 97.5 Num Measures Mean Num Trials
Ravens 0.875 0.845 0.904 1 18
Adaptive N Back 0.842 0.362 0.911 3 380
Kirby 0.835 0.746 0.897 12 14
Cognitive Reflection Survey 0.783 0.669 0.865 2 6
Threebytwo 0.772 0.694 0.845 2 383
Recent Probes 0.772 0.629 0.847 2 68
Stroop 0.76 0.445 0.844 7 67
Simple Reaction Time 0.738 0.668 0.826 1 149
Psychological Refractory Period Two Choices 0.732 0.499 0.85 4 186
Choice Reaction Time 0.727 0.506 0.848 2 150
Hierbackground-color:#eaeaea;hical Rule 0.707 0.592 0.792 3 298
Attention Network Task 0.703 -0.513 0.814 6 75
Simon 0.692 0.398 0.821 8 61
Directed Forgetting 0.681 0.479 0.782 2 67
Digit Span 0.675 0.544 0.786 2 10
Shape Matching 0.673 0.57 0.763 2 272
Information Sampling Task 0.67 0.118 0.859 8 10
Stop Signal 0.661 0.261 0.815 11 304
Spatial Span 0.658 0.382 0.794 2 10
Discount Titrate 0.645 0.496 0.753 1 36
Go Nogo 0.643 0.256 0.785 5 210
Keep Track 0.641 0.497 0.719 1 9
Tower Of London 0.634 0.436 0.855 4 12
Dot Pattern Expectancy 0.599 0.111 0.798 11 61
Columbia Card Task Hot 0.597 0.433 0.857 5 24
Stim Selective Stop Signal 0.577 0.375 0.752 4 104
Two Stage Decision 0.575 0.294 0.746 4 198
Local Global Letter 0.57 0.285 0.76 12 31
Dietary Decision 0.564 0.088 0.749 3 48
Columbia Card Task Cold 0.524 0.248 0.688 5 24
Holt Laury Survey 0.524 -0.122 0.659 3 10
Shift Task 0.476 -0.056 0.771 13 401
Angling Risk Task Always Sunny 0.428 0.039 0.636 6 21
Motor Selective Stop Signal 0.423 -0.357 0.798 4 63
Probabilistic Selection 0.402 -0.194 0.752 7 54
Writing Task 0.4 0.207 0.538 2 1
Bickel Titrator 0.066 -0.033 0.783 4 30

Number of trials

Does number of items in a task have a significant effect on the average ICC of (mostly) raw measures for all trials from a task? No. (no effect on Spearman either)

tmp = measure_labels %>%
  mutate(dv = as.character(dv)) %>%
  filter(task == 'task') %>%
  # left_join(rel_df[,c("dv", "spearman","icc")], by='dv') %>%
  left_join(boot_df[,c("dv", "spearman","icc")], by='dv') %>%
  filter(overall_difference != 'difference' & raw_fit %in% c('EZ', 'hddm') == FALSE)%>%
  separate(dv, c('task_name', 'extra_1', 'extra_2'), sep = '\\.',remove=FALSE) %>%
  select(-extra_1, -extra_2)

# summary(lm(icc ~ num_all_trials, data = tmp))
summary(lmerTest::lmer(icc ~ num_all_trials + (1|dv), data = tmp))
Linear mixed model fit by REML t-tests use Satterthwaite approximations
  to degrees of freedom [lmerMod]
Formula: icc ~ num_all_trials + (1 | dv)
   Data: tmp

REML criterion at convergence: -408213

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-12.548  -0.460   0.037   0.516   7.553 

Random effects:
 Groups   Name        Variance Std.Dev.
 dv       (Intercept) 0.04556  0.2135  
 Residual             0.00555  0.0745  
Number of obs: 174000, groups:  dv, 174

Fixed effects:
               Estimate Std. Error       df t value Pr(>|t|)    
(Intercept)    5.81e-01   2.11e-02 1.72e+02   27.60   <2e-16 ***
num_all_trials 2.12e-05   1.19e-04 1.72e+02    0.18     0.86    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Correlation of Fixed Effects:
            (Intr)
num_ll_trls -0.640
measure_labels %>%
  mutate(dv = as.character(dv)) %>%
  filter(task == 'task') %>%
  # left_join(rel_df[,c("dv", "spearman","icc")], by='dv') %>%
  left_join(boot_df[,c("dv", "spearman","icc")], by='dv') %>%
  filter(overall_difference != 'difference' & raw_fit %in% c('EZ', 'hddm') == FALSE)%>%
  separate(dv, c('task_name', 'extra_1', 'extra_2'), sep = '\\.',remove=FALSE) %>%
  select(-extra_1, -extra_2) %>%
  ggplot(aes(num_all_trials, icc))+
  geom_point()+
  geom_smooth(method="lm")+
  theme_bw()+
  xlab("Number of trials")+
  ylab("ICC")

Trial number dependence intrameasure

The above analysis was looking at the effect of number of trials across tasks. But some tasks might be bad for individual difference measurement regardless of how many trials there are in them whereas for others fewer trials might be yielding a sufficiently reliable measure.

For tasks for which dependent variables are estimated using many trials one can ask: Does the same measure get less reliable if fewer trials are used to estimate its reliability?

This won’t make sense for all tasks. For example to estimate a risk aversion parameter you need all trials for Holt and Laury. For Kirby and Bickel you have specific conditions looking at fewer trials. The Cognitive Reflection Task might be more appropriate to analyze each item seaprately. The writing task does not have trial numbers. For all others it might be interesting to investigate.

These kinds of analyses are too task-specific and in-depth for a paper that is trying to give a global sense of the differences between self-regulation measures in their suitablity for individual difference analyses based on their stability across time. Such analyses would provide a detailed examination of how to extract the most reliable/best individual difference measure from tasks with a set of mediocre variables to begin with. Though we do not provide such a comprehensive analysis of this sort in this paper we provide a single example of this approach and hope the open access we provide to the data spurs further work.

For this example we look at the retest reliability of the three dependent measures (average accuracy, median response time, total score) from the hierarchical rule task with 360 trials. Here is a graph of how the point estimates of the retest reliability changes for each of the dependent measures using different numbers of trials to estimate them. While the reliability estimate for each of the variables respectively are …, … and … using all of the trials, these values are independent of number of trials used in the task for only certain variables. Based on this analysis researchers might decide to use a version of the task with fewer trials or only measures with consistently high reliability estimates.

Example task: three by two

Post process dv’s from cluster to calculate reliabilities

t1_dvs = read.csv(paste0(retest_data_path, 't1_tbt_dvs.csv'))
t2_dvs = read.csv(paste0(retest_data_path, 't2_tbt_dvs.csv'))

t1_dvs = t1_dvs %>% select(-X)
t2_dvs = t2_dvs %>% select(-X)
hr_merge = merge(t1_dvs, t2_dvs, by = c("sub_id", "breaks"))

hr_merge = hr_merge %>%
  gather(key, value, -sub_id, -breaks) %>%
  separate(key, c("dv", "time"), sep="\\.") %>%
  mutate(time = ifelse(time == "x", 1, 2))

t1_dvs = hr_merge %>%
  filter(time == 1) %>%
  select(-time) %>%
  spread(dv, value)

t2_dvs = hr_merge %>%
  filter(time == 2) %>%
  select(-time) %>%
  spread(dv, value)

# calculate point estimates for reliability of each of the variables for each break
# get_icc for each break of tmp_t1_dvs and tmp_t2_dvs

trial_num_rel_df = data.frame(breaks=rep(NA, length(unique(t1_dvs$breaks))),
                              acc_icc=rep(NA, length(unique(t1_dvs$breaks))),
                              avg_rt_error_icc=rep(NA, length(unique(t1_dvs$breaks))),
                              std_rt_error_icc=rep(NA, length(unique(t1_dvs$breaks))),
                              avg_rt_icc=rep(NA, length(unique(t1_dvs$breaks))),
                              std_rt_icc=rep(NA, length(unique(t1_dvs$breaks))),
                              missed_percent_icc=rep(NA, length(unique(t1_dvs$breaks))),
                              cue_switch_cost_rt_100_icc=rep(NA, length(unique(t1_dvs$breaks))),
                              cue_switch_cost_rt_900_icc=rep(NA, length(unique(t1_dvs$breaks))),
                              task_switch_cost_rt_100_icc=rep(NA, length(unique(t1_dvs$breaks))),
                              task_switch_cost_rt_900_icc=rep(NA, length(unique(t1_dvs$breaks))),
                              cue_switch_cost_acc_100_icc=rep(NA, length(unique(t1_dvs$breaks))),
                              cue_switch_cost_acc_900_icc=rep(NA, length(unique(t1_dvs$breaks))),
                              task_switch_cost_acc_100_icc=rep(NA, length(unique(t1_dvs$breaks))),
                              task_switch_cost_acc_900_icc=rep(NA, length(unique(t1_dvs$breaks))))

for(i in 1:length(unique(t1_dvs$breaks))){
  cur_break = unique(t1_dvs$breaks)[i]
  tmp_t1_dvs = t1_dvs %>% filter(breaks == cur_break)
  tmp_t2_dvs = t2_dvs %>% filter(breaks == cur_break)
  trial_num_rel_df$breaks[i] = cur_break
  trial_num_rel_df$acc_icc[i] = get_icc("acc", tmp_t1_dvs, tmp_t2_dvs)
  trial_num_rel_df$avg_rt_error_icc[i] = get_icc("avg_rt_error", tmp_t1_dvs, tmp_t2_dvs)
  trial_num_rel_df$std_rt_error_icc[i] = get_icc("std_rt_error", tmp_t1_dvs, tmp_t2_dvs)
  trial_num_rel_df$avg_rt_icc[i] = get_icc("avg_rt", tmp_t1_dvs, tmp_t2_dvs)
  trial_num_rel_df$std_rt_icc[i] = get_icc("std_rt", tmp_t1_dvs, tmp_t2_dvs)
  trial_num_rel_df$missed_percent_icc[i] = get_icc("missed_percent", tmp_t1_dvs, tmp_t2_dvs)
  trial_num_rel_df$cue_switch_cost_rt_100_icc[i] = get_icc("cue_switch_cost_rt_100", tmp_t1_dvs, tmp_t2_dvs)
  trial_num_rel_df$cue_switch_cost_rt_900_icc[i] = get_icc("cue_switch_cost_rt_900", tmp_t1_dvs, tmp_t2_dvs)
  trial_num_rel_df$trial_switch_cost_rt_100_icc[i] = get_icc("task_switch_cost_rt_100", tmp_t1_dvs, tmp_t2_dvs)
  trial_num_rel_df$trial_switch_cost_rt_900_icc[i] = get_icc("task_switch_cost_rt_900", tmp_t1_dvs, tmp_t2_dvs)
  trial_num_rel_df$cue_switch_cost_acc_100_icc[i] = get_icc("cue_switch_cost_acc_100", tmp_t1_dvs, tmp_t2_dvs)
  trial_num_rel_df$cue_switch_cost_acc_900_icc[i] = get_icc("cue_switch_cost_acc_900", tmp_t1_dvs, tmp_t2_dvs)
  trial_num_rel_df$trial_switch_cost_acc_100_icc[i] = get_icc("task_switch_cost_acc_100", tmp_t1_dvs, tmp_t2_dvs)
  trial_num_rel_df$trial_switch_cost_acc_900_icc[i] = get_icc("task_switch_cost_acc_900", tmp_t1_dvs, tmp_t2_dvs)
}
rm(i, cur_break, tmp_t1_dvs, tmp_t2_dvs)

trial_num_rel_df$breaks = as.numeric(trial_num_rel_df$breaks)

write.csv(trial_num_rel_df, paste0(retest_data_path, 'trial_num_rel_df_tbt.csv'))
# trial_num_rel_df = read.csv(paste0(retest_data_path, 'trial_num_rel_df_tbt.csv'))

cols <- c("Accuracy" = '#084594', "Average RT error" = '#99000d', "SD RT error" = '#cb181d', "Average RT correct" = '#ef3b2c', "SD RT correct" = '#fb6a4a', "Missed percentage" = '#2171b5', "Cue switch cost RT (100)" = '#fc9272', "Cue switch cost RT (900)" = '#fcbba1', "Task switch cost RT (100)" = '#fee0d2', "Task switch cost RT (900)" = '#fff5f0', "Cue switch cost Acc (100)" = '#4292c6', "Cue switch cost Acc (900)" = '#6baed6', "Task switch cost Acc (100)" = '#9ecae1', "Task switch cost Acc (900)" = '#c6dbef')

trial_num_rel_df %>%
  gather(key, value, -breaks) %>%
  ggplot(aes(breaks*10, value, color=factor(key, levels = c("acc_icc", "avg_rt_error_icc", "std_rt_error_icc", "avg_rt_icc", "std_rt_icc", "missed_percent_icc", "cue_switch_cost_rt_100_icc", "cue_switch_cost_rt_900_icc", "task_switch_cost_rt_100_icc", "task_switch_cost_rt_900_icc", "cue_switch_cost_acc_100_icc", "cue_switch_cost_acc_900_icc", "task_switch_cost_acc_100_icc", "task_switch_cost_acc_900_icc"), labels = c("Accuracy", "Average RT error", "SD RT error", "Average RT correct", "SD RT correct", "Missed percentage", "Cue switch cost RT (100)", "Cue switch cost RT (900)", "Task switch cost RT (100)", "Task switch cost RT (900)", "Cue switch cost Acc (100)", "Cue switch cost Acc (900)", "Task switch cost Acc (100)", "Task switch cost Acc (900)"))))+
  geom_point()+
  geom_line()+
  theme_bw()+
  xlab("Number of trials")+
  ylab("ICC")+
  theme(legend.title = element_blank(),
        legend.position = 'bottom')+
  scale_color_manual(values = cols,
                     breaks = c("Average RT error", "Accuracy", "SD RT error", "Missed percentage", "Average RT correct", "Cue switch cost Acc (100)", "SD RT correct", "Cue switch cost Acc (900)", "Cue switch cost RT (100)", "Task switch cost Acc (100)", "Task switch cost RT (100)", "Task switch cost Acc (900)", "Task switch cost RT (900)"))+
  ylim(-1,1)+
  guides(color = guide_legend(ncol=2, byrow=T))

ggsave('Intrameasure_Trialnum_Dependendence.jpg', device = "jpeg", path = "/Users/zeynepenkavi/Dropbox/PoldrackLab/SRO_Retest_Analyses/output/figures", width = 10, height = 5, units = "in", dpi = 100)

Raw vs DDM

Checking DDM results in the bootstrapped estimates. Variables using all trials are significantly more reliable compared to difference scores. Raw measures don’t differ from DDM parameters. Which DDM is better depends on whether all trials are used.

tmp = measure_labels %>%
  mutate(dv = as.character(dv)) %>%
  filter(ddm_task == 1, 
         overall_difference != "condition",
         rt_acc != 'other') %>%
  drop_na() %>%
  left_join(boot_df[,c("dv", "icc", "spearman")], by = 'dv')

tmp %>%
  drop_na() %>% #try removing this in final release
  group_by(overall_difference, raw_fit, rt_acc) %>%
  summarise(icc_median = quantile(icc, probs = 0.5),
            icc_2.5 = quantile(icc, probs = 0.025),
            icc_97.5 = quantile(icc, probs = 0.975),
            spearman_median = quantile(spearman, probs = 0.5),
            spearman_2.5 = quantile(spearman, probs = 0.025),
            spearman_97.5 = quantile(spearman, probs = 0.975),
            num_vars = n()/1000) %>%
  datatable() %>%
  formatRound(columns=c('icc_median', 'icc_2.5', 'icc_97.5', 'spearman_median', 'spearman_2.5', 'spearman_97.5'), digits=3)
tmp_save = tmp %>%
  drop_na() %>% #try removing this in final release
  group_by(overall_difference, raw_fit, rt_acc) %>%
  summarise(icc_median = quantile(icc, probs = 0.5),
            icc_2.5 = quantile(icc, probs = 0.025),
            icc_97.5 = quantile(icc, probs = 0.975),
            num_vars = n()/1000) %>%
  ungroup() %>%
  mutate(overall_difference = as.character(overall_difference),
         raw_fit = as.character(raw_fit),
         rt_acc = as.character(rt_acc)) %>%
  arrange(-icc_median)

sjt.df(tmp_save %>% mutate_if(is.numeric, funs(round(., 3))), describe=F, hide.progress = TRUE, show.rownames = FALSE, file = "/Users/zeynepenkavi/Dropbox/PoldrackLab/SRO_Retest_Analyses/output/tables/ddm_rel_table.doc")
overall_difference raw_fit rt_acc icc_median icc_2.5 icc_97.5 num_vars
overall raw rt 0.736 0.535 0.854 16
overall EZ drift rate 0.724 0.528 0.826 14
overall raw accuracy 0.684 -0.375 0.832 14
overall hddm drift rate 0.684 0.387 0.816 16
overall EZ non-decision 0.647 0.197 0.794 14
overall EZ threshold 0.624 0.384 0.823 14
overall hddm threshold 0.544 0.316 0.668 14
overall hddm non-decision 0.449 -0.032 0.671 14
difference hddm drift rate 0.404 -0.242 0.651 18
difference raw rt 0.299 -0.193 0.63 25
difference raw accuracy 0.27 -0.245 0.698 13
difference EZ drift rate 0.248 -0.196 0.579 17
difference EZ non-decision 0.236 -0.204 0.652 17
difference EZ threshold 0.097 -0.449 0.537 17
difference hddm threshold 0.09 -0.201 0.345 1

Comparing overall vs difference: overall has higher reliability than difference.

summary(lmerTest::lmer(icc ~ overall_difference + (1|dv) ,tmp))
Linear mixed model fit by REML t-tests use Satterthwaite approximations
  to degrees of freedom [lmerMod]
Formula: icc ~ overall_difference + (1 | dv)
   Data: tmp

REML criterion at convergence: -401676

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-11.245  -0.467   0.041   0.521   6.005 

Random effects:
 Groups   Name        Variance Std.Dev.
 dv       (Intercept) 0.03486  0.1867  
 Residual             0.00966  0.0983  
Number of obs: 224000, groups:  dv, 224

Fixed effects:
                          Estimate Std. Error      df t value Pr(>|t|)    
(Intercept)                  0.245      0.018 221.400    13.6   <2e-16 ***
overall_differenceoverall    0.368      0.025 221.400    14.8   <2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Correlation of Fixed Effects:
            (Intr)
ovrll_dffrn -0.720

“consistent with the summing of variance in the difference score”

tmp = measure_labels %>%
  mutate(dv = as.character(dv)) %>%
  filter(ddm_task == 1, 
         overall_difference != "condition",
         rt_acc != 'other') %>%
  drop_na() %>%
  left_join(boot_df[,c("dv", "icc", "var_subs","var_ind", "var_resid")], by = 'dv') %>%
  mutate(var_subs_pct = var_subs/(var_subs+var_ind+var_resid),
         var_ind_pct = var_ind/(var_subs+var_ind+var_resid), 
         var_resid_pct = var_resid/(var_subs+var_ind+var_resid)) %>%
  select(-task, -ddm_task, -num_all_trials, -var_resid, -var_subs, -var_ind) %>%
  gather(key, value, -dv, -overall_difference, -raw_fit, -rt_acc, -icc)

tmp %>%
  filter(key == "var_resid_pct") %>%
  ggplot(aes(raw_fit, value, fill=overall_difference))+
  geom_boxplot()

summary(lmer(value  ~ overall_difference*raw_fit + (1|dv), data=tmp %>% filter(key=="var_resid_pct")))
Linear mixed model fit by REML ['lmerMod']
Formula: value ~ overall_difference * raw_fit + (1 | dv)
   Data: tmp %>% filter(key == "var_resid_pct")

REML criterion at convergence: -608085

Scaled residuals: 
   Min     1Q Median     3Q    Max 
-5.529 -0.543  0.020  0.601  5.547 

Random effects:
 Groups   Name        Variance Std.Dev.
 dv       (Intercept) 0.00539  0.0734  
 Residual             0.00385  0.0620  
Number of obs: 224000, groups:  dv, 224

Fixed effects:
                                      Estimate Std. Error t value
(Intercept)                             0.3437     0.0103    33.4
overall_differenceoverall              -0.1594     0.0153   -10.4
raw_fithddm                            -0.1070     0.0197    -5.4
raw_fitraw                             -0.0320     0.0157    -2.0
overall_differenceoverall:raw_fithddm   0.1270     0.0253     5.0
overall_differenceoverall:raw_fitraw    0.0240     0.0236     1.0

Correlation of Fixed Effects:
                        (Intr) ovrll_ rw_fth rw_ftr
ovrll_dffrn             -0.672                     
raw_fithddm             -0.521  0.350              
raw_fitraw              -0.653  0.439  0.340       
ovrll_dffrncvrll:rw_fth  0.406 -0.605 -0.780 -0.265
ovrll_dffrncvrll:rw_ftr  0.436 -0.649 -0.227 -0.668
                        ovrll_dffrncvrll:rw_fth
ovrll_dffrn                                    
raw_fithddm                                    
raw_fitraw                                     
ovrll_dffrncvrll:rw_fth                        
ovrll_dffrncvrll:rw_ftr  0.392                 
#Simple effects
summary(lmer(value  ~ overall_difference + (1|dv), data=tmp %>% filter(key=="var_resid_pct", raw_fit== "EZ")))
Linear mixed model fit by REML ['lmerMod']
Formula: value ~ overall_difference + (1 | dv)
   Data: tmp %>% filter(key == "var_resid_pct", raw_fit == "EZ")

REML criterion at convergence: -241956

Scaled residuals: 
   Min     1Q Median     3Q    Max 
-5.224 -0.536  0.041  0.607  4.544 

Random effects:
 Groups   Name        Variance Std.Dev.
 dv       (Intercept) 0.00448  0.0669  
 Residual             0.00431  0.0657  
Number of obs: 93000, groups:  dv, 93

Fixed effects:
                          Estimate Std. Error t value
(Intercept)                0.34371    0.00938    36.7
overall_differenceoverall -0.15937    0.01395   -11.4

Correlation of Fixed Effects:
            (Intr)
ovrll_dffrn -0.672
summary(lmer(value  ~ overall_difference + (1|dv), data=tmp %>% filter(key=="var_resid_pct", raw_fit== "raw")))
Linear mixed model fit by REML ['lmerMod']
Formula: value ~ overall_difference + (1 | dv)
   Data: tmp %>% filter(key == "var_resid_pct", raw_fit == "raw")

REML criterion at convergence: -184635

Scaled residuals: 
   Min     1Q Median     3Q    Max 
-4.952 -0.522  0.021  0.590  4.769 

Random effects:
 Groups   Name        Variance Std.Dev.
 dv       (Intercept) 0.00525  0.0724  
 Residual             0.00385  0.0620  
Number of obs: 68000, groups:  dv, 68

Fixed effects:
                          Estimate Std. Error t value
(Intercept)                 0.3117     0.0118   26.51
overall_differenceoverall  -0.1354     0.0177   -7.65

Correlation of Fixed Effects:
            (Intr)
ovrll_dffrn -0.664
summary(lmer(value  ~ overall_difference + (1|dv), data=tmp %>% filter(key=="var_resid_pct", raw_fit== "hddm")))
Linear mixed model fit by REML ['lmerMod']
Formula: value ~ overall_difference + (1 | dv)
   Data: tmp %>% filter(key == "var_resid_pct", raw_fit == "hddm")

REML criterion at convergence: -183232

Scaled residuals: 
   Min     1Q Median     3Q    Max 
-5.198 -0.592 -0.008  0.605  6.113 

Random effects:
 Groups   Name        Variance Std.Dev.
 dv       (Intercept) 0.00689  0.0830  
 Residual             0.00317  0.0563  
Number of obs: 63000, groups:  dv, 63

Fixed effects:
                          Estimate Std. Error t value
(Intercept)                 0.2368     0.0190   12.43
overall_differenceoverall  -0.0324     0.0228   -1.42

Correlation of Fixed Effects:
            (Intr)
ovrll_dffrn -0.836

Checking Rogosa’s claim that ‘Difference scores are reliable when individual differences in true change exist.’ How do we measure ‘individual difference in true change’ 1.between subject variability in difference score distributions 2.within subject variance in icc Using 2 (because comparable between subject variance across different measures would be hard) Expecting to see: positive correlation between mean icc and mean var_ind_pct Result: The correlation between mean icc and mean var ind pct is NOT significant BUT looking at the distribution of mean var ind pct there is not a lot of variability t

tmp = tmp %>%
  spread(key, value) %>%
  group_by(dv) %>%
  summarise(mean_var_ind_pct = mean(var_ind_pct),
            mean_var_subs_pct = mean(var_subs_pct),
            mean_icc = mean(icc)) %>%
  left_join(measure_labels, by='dv')
Warning: Column `dv` joining character vector and factor, coercing into
character vector
tmp %>%
  filter(overall_difference == 'difference') %>%
  ggplot(aes(mean_var_ind_pct, mean_icc))+
  geom_point()+
  geom_abline(slope=1, intercept=0)+
  geom_smooth(method = "lm")

summary(lm(mean_icc ~ mean_var_ind_pct,data=tmp %>%
  filter(overall_difference == 'difference')))

Call:
lm(formula = mean_icc ~ mean_var_ind_pct, data = tmp %>% filter(overall_difference == 
    "difference"))

Residuals:
    Min      1Q  Median      3Q     Max 
-0.6262 -0.1294 -0.0076  0.1613  0.3764 

Coefficients:
                 Estimate Std. Error t value Pr(>|t|)    
(Intercept)        0.2302     0.0366    6.29  7.1e-09 ***
mean_var_ind_pct   0.0587     0.1197    0.49     0.62    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 0.212 on 106 degrees of freedom
Multiple R-squared:  0.00227,   Adjusted R-squared:  -0.00715 
F-statistic: 0.241 on 1 and 106 DF,  p-value: 0.625
tmp %>%
  filter(overall_difference == 'difference') %>%
  ggplot(aes(mean_var_ind_pct))+
  geom_density()

summary(tmp$mean_var_ind_pct[tmp$overall_difference == "difference"])
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
 0.0917  0.1350  0.1820  0.2540  0.3350  0.9560 
summary(tmp$mean_var_subs_pct[tmp$overall_difference == "difference"])
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
 0.0255  0.3690  0.4440  0.4330  0.5170  0.6300 
tmp = tmp %>% filter(overall_difference == 'difference')
t.test(tmp$mean_var_ind_pct, tmp$mean_var_subs_pct, paired=T)

    Paired t-test

data:  tmp$mean_var_ind_pct and tmp$mean_var_subs_pct
t = -6.7, df = 110, p-value = 8e-10
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -0.2318 -0.1265
sample estimates:
mean of the differences 
                -0.1791 
tmp = measure_labels %>%
  mutate(dv = as.character(dv)) %>%
  filter(ddm_task == 1, 
         overall_difference != "condition",
         rt_acc != 'other') %>%
  drop_na() %>%
  left_join(boot_df[,c("dv", "icc", "spearman")], by = 'dv')

Comparing raw vs ddm in overall estimates: EZ is significantly better than HDDM and comparable to raw estimates.

summary(lmerTest::lmer(icc ~ raw_fit + (1|dv) ,tmp %>% filter(overall_difference == "overall")))
Linear mixed model fit by REML t-tests use Satterthwaite approximations
  to degrees of freedom [lmerMod]
Formula: icc ~ raw_fit + (1 | dv)
   Data: tmp %>% filter(overall_difference == "overall")

REML criterion at convergence: -278565

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-11.366  -0.485   0.031   0.510   8.141 

Random effects:
 Groups   Name        Variance Std.Dev.
 dv       (Intercept) 0.02299  0.1516  
 Residual             0.00526  0.0725  
Number of obs: 116000, groups:  dv, 116

Fixed effects:
            Estimate Std. Error       df t value Pr(>|t|)    
(Intercept)   0.6493     0.0234 113.4000   27.75   <2e-16 ***
raw_fithddm  -0.1068     0.0327 113.4000   -3.27   0.0014 ** 
raw_fitraw    0.0186     0.0362 113.4000    0.51   0.6080    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Correlation of Fixed Effects:
            (Intr) rw_fth
raw_fithddm -0.715       
raw_fitraw  -0.645  0.462

Comparing raw vs ddm in difference scores: EZ is significantly worse than HDDM and comparable to raw estimates.

summary(lmerTest::lmer(icc ~ raw_fit + (1|dv) ,tmp %>% filter(overall_difference == "difference")))
Linear mixed model fit by REML t-tests use Satterthwaite approximations
  to degrees of freedom [lmerMod]
Formula: icc ~ raw_fit + (1 | dv)
   Data: tmp %>% filter(overall_difference == "difference")

REML criterion at convergence: -150642

Scaled residuals: 
   Min     1Q Median     3Q    Max 
-9.214 -0.535  0.065  0.615  4.172 

Random effects:
 Groups   Name        Variance Std.Dev.
 dv       (Intercept) 0.0423   0.206   
 Residual             0.0144   0.120   
Number of obs: 108000, groups:  dv, 108

Fixed effects:
            Estimate Std. Error       df t value Pr(>|t|)    
(Intercept)   0.1900     0.0288 105.1000    6.59  1.8e-09 ***
raw_fithddm   0.1425     0.0553 105.1000    2.58    0.011 *  
raw_fitraw    0.0855     0.0441 105.1000    1.94    0.055 .  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Correlation of Fixed Effects:
            (Intr) rw_fth
raw_fithddm -0.521       
raw_fitraw  -0.653  0.340
tmp %>%
  ggplot(aes(factor(raw_fit, levels = c("raw", "EZ", "hddm"), labels=c("Raw", "EZ-diffusion", "Hierarchical diffusion")), icc, fill=factor(rt_acc, levels = c("rt","accuracy", "drift rate", "threshold", "non-decision"), labels=c("Response Time", "Accuracy","Drift Rate", "Threshold", "Non-decision"))))+
  geom_boxplot()+
  facet_wrap(~factor(overall_difference, levels=c("overall", "difference"), labels=c("Overall", "Difference")))+
  theme_bw()+
  ylab("ICC")+
  xlab("")+
  theme(legend.title = element_blank(),
        legend.position = 'bottom',
        legend.text = element_text(size=16),
        strip.text = element_text(size=16),
        axis.text = element_text(size = 16),
        text = element_text(size=16))+
  guides(fill = guide_legend(ncol = 2))+
  scale_fill_discrete(breaks=c("Drift Rate", "Threshold", "Non-decision", "Response Time", "Accuracy"))

ggsave('Bootstrap_DDM_Comp.jpg', device = "jpeg", path = "/Users/zeynepenkavi/Dropbox/PoldrackLab/SRO_Retest_Analyses/output/figures/", width = 14, height = 10, units = "in", limitsize = FALSE)
tmp2 = measure_labels %>%
  mutate(dv = as.character(dv)) %>%
  filter(ddm_task == 1, 
         overall_difference != "condition",
         rt_acc != "other") %>%
  drop_na() %>%
  left_join(rel_df[,c("dv", "icc", "spearman")], by = 'dv')

tmp2 %>%
  ggplot(aes(factor(raw_fit, levels = c("raw", "EZ", "hddm"), labels=c("Raw", "EZ-diffusion", "Hierarchical diffusion")), icc, fill=factor(rt_acc, levels = c("rt","accuracy", "drift rate", "threshold", "non-decision"), labels=c("Response Time", "Accuracy","Drift Rate", "Threshold", "Non-decision"))))+
  geom_boxplot()+
  facet_wrap(~factor(overall_difference, levels=c("overall", "difference"), labels=c("Overall", "Difference")))+
  theme_bw()+
  ylab("ICC")+
  xlab("")+
  theme(legend.title = element_blank(),
        legend.position = 'bottom',
        legend.text = element_text(size=16),
        strip.text = element_text(size=16),
        axis.text = element_text(size = 16),
        text = element_text(size=16))+
  guides(fill = guide_legend(ncol = 2))+
  scale_fill_discrete(breaks=c("Drift Rate", "Threshold", "Non-decision", "Response Time", "Accuracy"))

ggsave('PointEst_DDM_Comp.jpg', device = "jpeg", path = "/Users/zeynepenkavi/Dropbox/PoldrackLab/SRO_Retest_Analyses/output/figures/", width = 14, height = 10, units = "in", limitsize = FALSE)

Survey reliabilities

tmp = measure_labels %>%
  mutate(dv = as.character(dv)) %>%
  filter(task == 'survey') %>%
  left_join(boot_df[,c("dv", "icc", "spearman")], by = 'dv') %>%
  filter(overall_difference != 'difference' & raw_fit %in% c('EZ', 'hddm') == FALSE)%>%
  separate(dv, c('task_name', 'extra_1', 'extra_2'), sep = '\\.',remove=FALSE) %>%
  select(-extra_1, -extra_2) %>%
  group_by(task_name) %>%
  summarise(median_icc = median(icc),
            median_spearman = median(spearman),
            icc_2.5 = quantile(icc, probs = 0.025),
            icc_97.5 = quantile(icc, probs = 0.975),
            min_spearman = min(spearman),
            max_spearman = max(spearman),
            num_measures = n()/1000,
            mean_num_trials = round(mean(num_all_trials)))%>%
  arrange(-median_icc, -median_spearman)

tmp %>%
  datatable() %>%
  formatRound(columns=c('median_spearman', 'median_icc',
                        'min_spearman', 'icc_2.5',
                        'max_spearman', 'icc_97.5'), digits=3)
tmp = tmp%>%
  select(-min_spearman, -max_spearman,-median_spearman) %>%
  mutate(task_name = gsub("_", " ", task_name),
         task_name = gsub("(^|[[:space:]])([[:alpha:]])", "\\1\\U\\2", task_name, perl=TRUE),
         task_name = gsub("Survey", "", task_name))

names(tmp) = gsub("_", " ", names(tmp))
names(tmp) = gsub("(^|[[:space:]])([[:alpha:]])", "\\1\\U\\2", names(tmp), perl=TRUE)

sjt.df(tmp %>% mutate_if(is.numeric, funs(round(., 3))), describe=F, hide.progress = TRUE, show.rownames = FALSE, file = "/Users/zeynepenkavi/Dropbox/PoldrackLab/SRO_Retest_Analyses/output/tables/survey_rel_table.doc")
Task Name Median Icc Icc 2.5 Icc 97.5 Num Measures Mean Num Trials
Self Regulation 0.95 0.936 0.961 1 31
Grit Scale 0.942 0.924 0.958 1 8
Brief Self Control 0.941 0.92 0.966 1 13
Ten Item Personality 0.936 0.852 0.966 5 2
Time Perspective 0.92 0.865 0.954 5 11
Five Facet Mindfulness 0.909 0.828 0.965 6 13
Bis Bas 0.904 0.851 0.958 5 7
Dospert Rt 0.903 0.855 0.953 5 6
Impulsive Venture 0.9 0.815 0.95 2 17
Sensation Seeking 0.899 0.852 0.951 5 16
Mindful Attention Awareness 0.897 0.862 0.928 1 15
Erq 0.896 0.827 0.933 2 5
Upps Impulsivity 0.886 0.718 0.959 5 12
Mpq Control 0.879 0.796 0.95 1 24
Eating 0.872 0.813 0.914 4 9
Bis11 0.864 0.63 0.939 4 15
Dickman 0.85 0.721 0.91 2 12
Future Time Perspective 0.844 0.792 0.903 1 10
Dospert Rp 0.842 0.761 0.901 5 6
Dospert Eb 0.8 0.711 0.868 5 6
Selection Optimization Compensation 0.799 0.598 0.898 4 12
Leisure Time Activity 0.798 0.731 0.857 1 1
Theories Of Willpower 0.794 0.725 0.86 1 12

Number of items

Does number of items in a survey have a significant effect on the average ICC of survey measures? No.

tmp = measure_labels %>%
  mutate(dv = as.character(dv)) %>%
  filter(task == 'survey') %>%
  # left_join(rel_df[,c("dv", "spearman","icc")], by='dv') %>%
  left_join(boot_df[,c("dv", "spearman","icc")], by='dv') %>%
  filter(overall_difference != 'difference' & raw_fit %in% c('EZ', 'hddm') == FALSE)%>%
  separate(dv, c('task_name', 'extra_1', 'extra_2'), sep = '\\.',remove=FALSE) %>%
  select(-extra_1, -extra_2)

# summary(lm(icc ~ num_all_trials, data = tmp))
summary(lmerTest::lmer(icc ~ num_all_trials + (1|dv), data = tmp))
Linear mixed model fit by REML t-tests use Satterthwaite approximations
  to degrees of freedom [lmerMod]
Formula: icc ~ num_all_trials + (1 | dv)
   Data: tmp

REML criterion at convergence: -320933

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-10.265  -0.516  -0.004   0.540   6.540 

Random effects:
 Groups   Name        Variance Std.Dev.
 dv       (Intercept) 0.003327 0.0577  
 Residual             0.000673 0.0259  
Number of obs: 72000, groups:  dv, 72

Fixed effects:
                Estimate Std. Error        df t value Pr(>|t|)    
(Intercept)     0.864206   0.011510 70.400000   75.08   <2e-16 ***
num_all_trials  0.001071   0.000915 70.400000    1.17     0.25    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Correlation of Fixed Effects:
            (Intr)
num_ll_trls -0.807
measure_labels %>%
  mutate(dv = as.character(dv)) %>%
  filter(task == 'task') %>%
  # left_join(rel_df[,c("dv", "spearman","icc")], by='dv') %>%
  left_join(boot_df[,c("dv", "spearman","icc")], by='dv') %>%
  filter(overall_difference != 'difference' & raw_fit %in% c('EZ', 'hddm') == FALSE)%>%
  separate(dv, c('task_name', 'extra_1', 'extra_2'), sep = '\\.',remove=FALSE) %>%
  select(-extra_1, -extra_2) %>%
  ggplot(aes(num_all_trials, icc))+
  geom_point()+
  geom_smooth(method="lm")+
  theme_bw()+
  xlab("Number of trials")+
  ylab("ICC")

LS0tCnRpdGxlOiAnU2VsZiBSZWd1bGF0aW9uIE9udG9sb2d5IFJldGVzdCBBbmFseXNlcycKb3V0cHV0OgpnaXRodWJfZG9jdW1lbnQ6CnRvYzogeWVzCnRvY19mbG9hdDogeWVzCi0tLQoKYGBge3IsIG1lc3NhZ2U9RkFMU0UsIHdhcm5pbmc9RkFMU0UsIGluY2x1ZGU9RkFMU0V9CnNvdXJjZSgnL1VzZXJzL3pleW5lcGVua2F2aS9Ecm9wYm94L1BvbGRyYWNrTGFiL1NST19SZXRlc3RfQW5hbHlzZXMvY29kZS9TUk9fUmV0ZXN0X0FuYWx5c2VzX0hlbHBlcl9GdW5jdGlvbnMuUicpCnRoZW1lX3NldCh0aGVtZV9idygpKQoKdGVzdF9kYXRhX3BhdGggPSAnL1VzZXJzL3pleW5lcGVua2F2aS9Eb2N1bWVudHMvUG9sZHJhY2tMYWJMb2NhbC9TZWxmX1JlZ3VsYXRpb25fT250b2xvZ3kvRGF0YS9Db21wbGV0ZV8wMy0yOS0yMDE4LycKCnJldGVzdF9kYXRhX3BhdGggPSAnL1VzZXJzL3pleW5lcGVua2F2aS9Eb2N1bWVudHMvUG9sZHJhY2tMYWJMb2NhbC9TZWxmX1JlZ3VsYXRpb25fT250b2xvZ3kvRGF0YS9SZXRlc3RfMDMtMjktMjAxOC8nCmBgYAoKIyBJbnRyb2R1Y3Rpb24KClRoZSBwc3ljaG9sb2dpY2FsIGNvbnN0cnVjdCBvZiBzZWxmLXJlZ3VsYXRpb24sIG9yIHJlbGF0ZWQgY29uY2VwdHMgb2YgaW1wdWxzaXZpdHksIHNlbGYtY29udHJvbCwgaW5oaWJpdGlvbiBhbmQgb3RoZXJzIGNvcnJlbGF0ZSB3aXRoIHByb2JsZW1hdGljIHJlYWwtd29ybGQgYmVoYXZpb3JzIHN1Y2ggYXMgZGlzb3JkZXJlZCBlYXRpbmcgKE1vYmJzIGV0IGFsLiwgMjAxMCwgVmVyYmVrZW4gZXQgYWwuLCAyMDA5LCBOZWRlcmtvb3JuIGV0IGFsLiAyMDA2YSwgTmVkZXJrb29ybiBldCBhbC4sIDIwMDZiKSwgZ2FtYmxpbmcgKExhd3JlbmNlIGV0IGFsLiwgMjAwOSwgRnVlbnRlcyBldCBhbC4sIDIwMDYsIEFsZXNzaSBhbmQgUGV0cnksIDIwMDMpLCBkcnVnIGFkZGljdGlvbiAoQ29mZmV5IGV0IGFsLiwgMjAwMywgZGUgV2l0LCBDcmVhbiBhbmQgSm9obiwgMjAwMCwgU2hlciwgQmFydGhvbG93IGFuZCBXb29kLCAyMDAwLCBLaXJieSwgUGV0cnkgYW5kIEJpY2tlbCwgMTk5OSkgb3IgYmFkIGZpbmFuY2lhbCBkZWNpc2lvbnMgKE1laWVyIGFuZCBTcHJlbmdlciwgMjAxNSwgTWVpZXIgYW5kIFNwcmVuZ2VyLCAyMDEzLCBNZWllciBhbmQgU3ByZW5nZXIgMjAxMikuIEluIHRoZXNlIGxpbmVzIG9mIHdvcmsgbWVhc3VyZXMgb2Ygc2VsZi1yZWd1bGF0aW9uIGFyZSB1c2VkIGFzIGJlaGF2aW9yYWwgYXNzYXlzIGZvciBpbmRpdmlkdWFsIGRpZmZlcmVuY2UgYW5hbHlzZXMuIEFuIHVuZGVybHlpbmcgYXNzdW1wdGlvbiBvZiB0cmVhdGluZyBiZWhhdmlvcmFsIG1lYXN1cmVzIGFzIHJlZmxlY3RpdmUgb2YgcGVyc29uLXNwZWNpZmljIGNoYXJhY3RlcmlzdGljcyBpcyB0aGF0IHRoZSBtZWFzdXJlcyBvZiBzZWxmLXJlZ3VsYXRpb24gYXJlIHN0YWJsZSBhY3Jvc3MgdGltZS4gSW4gb3RoZXIgd29yZHMsIHRoYXQgdGhleSBoYXZlIGhpZ2ggdGVzdC1yZXRlc3QgcmVsaWFiaWxpdHkgKG9yIGhpZ2ggYmV0d2Vlbi0gYW5kIGxvdyB3aXRoaW4tc3ViamVjdCB2YXJpYWJpbGl0eSkuIFRoaXMgYXNzdW1wdGlvbiBpcyBub3QgZXh0ZW5zaXZlbHkgYW5kIGVxdWFsbHkgdGVzdGVkIGluIHRoZSBsaXRlcmF0dXJlIGZvciBhbGwgbWVhc3VyZXMgb2Ygc2VsZi1yZWd1bGF0aW9uLiAgICAKU2VsZi1yZWd1bGF0aW9uIG1lYXN1cmVzIGNhbiBiZSBncm91cGVkIGluIHR3byBicm9hZCBjYXRlZ29yaWVzOiBTdXJ2ZXlzIGFuZCBjb2duaXRpdmUgdGFza3MuIFdoaWxlIHRoZSBmb3JtZXIgdHlwaWNhbGx5IGdvZXMgdGhyb3VnaCBzb21lIGZvcm0gb2YgcHN5Y2hvbWV0cmljIHRlc3Rpbmcgb2Z0ZW4gZmlsdGVyaW5nIG91dCBpdGVtcyB3aXRoIGxvdyB0ZXN0LXJldGVzdCByZWxpYWJpbGl0eSB0aGlzIGlzIGxlc3MgZnJlcXVlbnRseSB0cnVlIGZvciB0aGUgbGF0dGVyLiBUaG91Z2ggc29tZSBlbXBpcmljYWwgcmVzdWx0cyBvbiB0ZXN0LXJldGVzdCByZWxpYWJpbGl0aWVzIG9mIGNlcnRhaW4gY29nbml0aXZlIHRhc2sgbWVhc3VyZXMgYXJlIHJlcG9ydGVkIGluIHBvY2tldHMgb2YgdGhlIGxpdGVyYXR1cmUgYW4gZXhoYXVzdGl2ZSBzdW1tYXJ5IG9yIHN5c3RlbWF0aWMgZWxpbWluYXRpb24gb2YgdGFza3Mgb3IgbWVhc3VyZXMgYmFzZWQgb24gc3RhYmlsaXR5IGFjcm9zcyB0aW1lIGRvZXMgbm90IHNlZW0gdG8gZXhpc3QuIFRlc3QtcmV0ZXN0IHJlbGlhYmlsaXR5IGlzLCBob3dldmVyLCBjcnVjaWFsIGZvciBpbmRpdmlkdWFsIGRpZmZlcmVuY2UgYW5hbHlzZXMgKEhlZGdlLCBQb3dlbGwgYW5kIFN1bW5lciwgMjAxNykuICAgClRvIGFuc3dlciB0aGUgcXVlc3Rpb24gb2YgcmVsaWFiaWxpdHkgb2Ygc2VsZi1yZWd1bGF0aW9uIG1lYXN1cmVzIGNvbXByZWhlbnNpdmVseSB3ZSBjcmVhdGVkIGEgbGFyZ2UgYmF0dGVyeSBjb25zaXN0aW5nIG9mIGJvdGggY29nbml0aXZlIHRhc2sgYW5kIHN1cnZleSBtZWFzdXJlcy4gSW4gdGhpcyBwYXBlciB3ZSBtYWtlIHR3byBjb250cmlidXRpb25zOiBGaXJzdCB3ZSBwcm92aWRlIGEgcmV2aWV3IG9mIHRoZXNlIG1lYXN1cmVzIGFsb25nIHdpdGggdGhlaXIgcmVwb3J0ZWQgcmVsaWFiaWxpdGllcyB3aGVuIGF2YWlsYWJsZS4gVGhlbiB3ZSByZXBvcnQgcmVzdWx0cyBvZiBhIGxhcmdlIHN0dWR5IHdlIGNvbmR1Y3RlZCB3aGVyZSB3ZSBjb2xsZWN0ZWQgcmV0ZXN0IHJlbGlhYmlsaXR5IGRhdGEgdXNpbmcgYWxsIG9mIHRoZXNlIG1lYXN1cmVzLiBUaGlzIGVmZm9ydCBub3Qgb25seSBmaWxscyBpbiBhIG5vdGFibGUgZ2FwIGluIHRoZSBsaXRlcmF0dXJlIGluIGNsYXJpZnlpbmcgd2hpY2ggbWVhc3VyZXMgb2Ygc2VsZi1yZWd1bGF0aW9uIGFyZSBzdGFibGUgYWNyb3NzIHRpbWUgYnV0IGFsc28gcHJvdmlkZXMgZ3VpZGFuY2Ugb24gZmFjdG9ycyB0byBwYXkgYXR0ZW50aW9uIHRvIHdoZW4gY29uc3RydWN0aW5nIG5ldyB0YXNrcyBmb3IgaW5kaXZpZHVhbCBkaWZmZXJlbmNlIG1lYXN1cmVzLiAgCgojIE1ldGhvZHMKCiMjIERhdGEgY29sbGVjdGlvbgoKVGhpcyBzYW1wbGUgY29uc2lzdHMgb2YgZGF0YSBmb3IgMTUwIHN1YmplY3RzIG9mIHRoZSBvcmlnaW5hbCBzYW1wbGUgb2YgNTIyIHRoYXQgaGFzIGNvbXBsZXRlZCB0aGUgaW5pdGlhbCBiYXR0ZXJ5IG9mIDM3IGNvZ25pdGl2ZSB0YXNrcywgMjMgc3VydmV5cyBhbmQgMyBzdXJ2ZXlzIG9uIGRlbW9ncmFwaGljcy4gRGV0YWlscyBvZiB0aGUgb3JpZ2luYWwgc2FtcGxlIGFzIHdlbGwgYXMgcXVhbGl0eSBjb250cm9sIChxYykgcHJvY2VkdXJlcyBhcmUgZGVzY3JpYmVkIGVsc2V3aGVyZSAoRWlzZW5iZXJnIGV0IGFsLiwgMjAxNykuIEludml0ZWQgcGFydGljaXBhbnRzIHdlcmUgY2hvc2VuIHJhbmRvbWx5IGFuZCBvbmx5IHN1YnNldHMgb2YgdGhlbSB3ZXJlIGludml0ZWQgZm9yIGEgZ2l2ZW4gYmF0Y2ggKGluc3RlYWQgb2Ygb3BlbmluZyB0aGUgYmF0dGVyeSB0byBhbGwgcXVhbGlmaWVkIHN1YmplY3RzKSB3aXRoIHRoZSBpbnRlbnRpb24gdG8gYXZvaWQgYSBwb3RlbnRpYWwgb3ZlcnNhbXBsaW5nIGFuZCBiaWFzIHRvd2FyZHMgImhpZ2ggc2VsZiByZWd1bGF0b3JzIi4KCmBgYHtyIHdhcm5pbmc9RkFMU0UsIG1lc3NhZ2U9RkFMU0V9CndvcmtlcnMgPSByZWFkLmNzdihwYXN0ZTAocmV0ZXN0X2RhdGFfcGF0aCwnTG9jYWwvVXNlcl83MTc1NzBfd29ya2Vycy5jc3YnKSkKd29ya2VycyA9IHdvcmtlcnMgJT4lIAogIGdyb3VwX2J5KFdvcmtlci5JRCkgJT4lCiAgbXV0YXRlKFJldGVzdF93b3JrZXI9aWZlbHNlKHN1bShDVVJSRU5ULlJldGVzdFdvcmtlcixDVVJSRU5ULlJldGVzdFdvcmtlckIyLENVUlJFTlQuUmV0ZXN0V29ya2VyQjMsQ1VSUkVOVC5SZXRlc3RXb3JrZXJCNCxDVVJSRU5ULlJldGVzdFdvcmtlckI1LG5hLnJtPVQpPjAsMSwwKSkgJT4lCiAgdW5ncm91cCgpCgp3b3JrZXJfY291bnRzIDwtIGZyb21KU09OKHBhc3RlMChyZXRlc3RfZGF0YV9wYXRoLCcvTG9jYWwvcmV0ZXN0X3dvcmtlcl9jb3VudHMuanNvbicpKQoKd29ya2VyX2NvdW50cyA9IGFzLmRhdGEuZnJhbWUodW5saXN0KHdvcmtlcl9jb3VudHMpKQpuYW1lcyh3b3JrZXJfY291bnRzKSA9ICJ0YXNrX2NvdW50IgpgYGAKCkluIHRvdGFsIGByIHN1bSh3b3JrZXJzJFJldGVzdF93b3JrZXIpYCBwYXJ0aWNpcGFudHMgd2VyZSBpbnZpdGVkLCBgciBucm93KHdvcmtlcl9jb3VudHMpYCBiZWdhbiB0aGUgYmF0dGVyeSwgYHIgc3VtKHdvcmtlcl9jb3VudHMkdGFza19jb3VudCA+PSA2MilgIGNvbXBsZXRlZCB0aGUgYmF0dGVyeSBhbmQgMTUwIHByb3ZpZGVkIGRhdGEgdGhhdCBwYXNzZWQgcWMgZm9yIGJvdGggdGltZSBwb2ludHMuIE91ciB0YXJnZXQgc2FtcGxlIHNpemUgd2FzIGRldGVybWluZWQgaW4gYWR2YW5jZSBvZiBkYXRhIGNvbGxlY3Rpb24gYW5kIGRhdGEgY29sbGVjdGlvbiBjb250aW51ZWQgdW50aWwgdGhpcyBudW1iZXIgb2YgcGFydGljaXBhbnRzIHdpdGggZGF0YSB0aGF0IHN1cnZpdmVkIHFjIHdhcyByZWFjaGVkLgoKYGBge3Igd2FybmluZz1GQUxTRSwgbWVzc2FnZT1GQUxTRX0KZGlzY19jb21wX2RhdGUgPSByZWFkLmNzdihwYXN0ZTAocmV0ZXN0X2RhdGFfcGF0aCwnTG9jYWwvZGlzY292ZXJ5X2NvbXBsZXRpb25fZGF0ZXMuY3N2JyksIGhlYWRlcj1GQUxTRSkKdmFsX2NvbXBfZGF0ZSA9IHJlYWQuY3N2KHBhc3RlMChyZXRlc3RfZGF0YV9wYXRoLCdMb2NhbC92YWxpZGF0aW9uX2NvbXBsZXRpb25fZGF0ZXMuY3N2JyksIGhlYWRlcj1GQUxTRSkKdGVzdF9jb21wX2RhdGUgPSByYmluZChkaXNjX2NvbXBfZGF0ZSwgdmFsX2NvbXBfZGF0ZSkKcm0oZGlzY19jb21wX2RhdGUsIHZhbF9jb21wX2RhdGUpCnJldGVzdF9jb21wX2RhdGUgPSByZWFkLmNzdihwYXN0ZTAocmV0ZXN0X2RhdGFfcGF0aCwnTG9jYWwvcmV0ZXN0X2NvbXBsZXRpb25fZGF0ZXMuY3N2JyksIGhlYWRlcj1GQUxTRSkKY29tcF9kYXRlcyA9IG1lcmdlKHJldGVzdF9jb21wX2RhdGUsIHRlc3RfY29tcF9kYXRlLCBieT0iVjEiKQpuYW1lcyhjb21wX2RhdGVzKSA8LSBjKCJzdWJfaWQiLCAicmV0ZXN0X2NvbXAiLCAidGVzdF9jb21wIikKY29tcF9kYXRlcyRyZXRlc3RfY29tcCA9IGFzLkRhdGUoY29tcF9kYXRlcyRyZXRlc3RfY29tcCkKY29tcF9kYXRlcyR0ZXN0X2NvbXAgPSBhcy5EYXRlKGNvbXBfZGF0ZXMkdGVzdF9jb21wKQpjb21wX2RhdGVzJGRheXNfYnR3ID0gd2l0aChjb21wX2RhdGVzLCByZXRlc3RfY29tcC10ZXN0X2NvbXApCmBgYAoKRGF0YSBjb2xsZWN0aW9uIHRvb2sgcGxhY2Ugb24gYXZlcmFnZSBgciByb3VuZChtZWFuKGFzLm51bWVyaWMoY29tcF9kYXRlcyRkYXlzX2J0dykpKWAgbnVtYmVyIG9mIGRheXMgYWZ0ZXIgdGhlIGNvbXBsZXRpb24gb2YgdGhlIGluaXRpYWwgYmF0dGVyeSB3aXRoIGEgcmFuZ2Ugb2YgYHIgcm91bmQocmFuZ2UoYXMubnVtZXJpYyhjb21wX2RhdGVzJGRheXNfYnR3KSkpWzFdYCB0byBgciByb3VuZChyYW5nZShhcy5udW1lcmljKGNvbXBfZGF0ZXMkZGF5c19idHcpKSlbMl1gIGRheXMuCgpgYGB7cn0Kcm0odGVzdF9jb21wX2RhdGUsIHJldGVzdF9jb21wX2RhdGUsIGNvbXBfZGF0ZXMpCmBgYAoKIyMgRGVtb2dyYXBoaWNzCgpgYGB7cn0KdGVzdF9kZW1vZyA8LSByZWFkLmNzdihwYXN0ZTAocmV0ZXN0X2RhdGFfcGF0aCwgJy90MV9kYXRhL2RlbW9ncmFwaGljX2hlYWx0aC5jc3YnKSkKCnJldGVzdF9kZW1vZyA8LSByZWFkLmNzdihwYXN0ZTAocmV0ZXN0X2RhdGFfcGF0aCwgJ2RlbW9ncmFwaGljX2hlYWx0aC5jc3YnKSkKCnJldGVzdF9kZW1vZyA9IHJldGVzdF9kZW1vZ1tyZXRlc3RfZGVtb2ckWCAlaW4lIHRlc3RfZGVtb2ckWCxdCgpuYW1lcyh0ZXN0X2RlbW9nKVt3aGljaChuYW1lcyh0ZXN0X2RlbW9nKSA9PSAnWCcpXSA8LSdzdWJfaWQnCm5hbWVzKHJldGVzdF9kZW1vZylbd2hpY2gobmFtZXMocmV0ZXN0X2RlbW9nKSA9PSAnWCcpXSA8LSdzdWJfaWQnCgpzdW1tYXJ5KHRlc3RfZGVtb2cgJT4lCiAgICAgICAgICBzZWxlY3QoU2V4LCBBZ2UpKQoKc3VtbWFyeShyZXRlc3RfZGVtb2cgJT4lCiAgICAgICAgICBzZWxlY3QoU2V4LCBBZ2UpKQpgYGAKCiMjIExpdGVyYXR1cmUKCk9uZSBvZiB0aGUgbWFqb3IgY29udHJpYnV0aW9ucyBvZiB0aGlzIHByb2plY3QgaXMgYSBjb21wcmVoZW5zaXZlIGxpdGVyYXR1cmUgcmV2aWV3IG9mIHRoZSByZXRlc3QgcmVsaWFiaWxpdGllcyBvZiB0aGUgc3VydmV5cyBhbmQgdGFza3MgdGhhdCB3ZXJlIHVzZWQuIFdlIHJldmlld2VkIHRoZSBsaXRlcmF0dXJlIG9uIGEgbWVhc3VyZSAoYXMgb3Bwb3NlZCB0byB0YXNrIGxldmVsKSBwYXlpbmcgYXR0ZW50aW9uIHRvIGRpZmZlcmVuY2VzIGluIHNhbXBsZSBzaXplLCB0aGUgZGVsYXkgYmV0d2VlbiB0aGUgdHdvIG1lYXN1cmVtZW50cyBhcyB3ZWxsIGFzIHRoZSBzdGF0aXN0aWMgdGhhdCB3YXMgdXNlZCB0byBhc3Nlc3MgcmVsaWFiaWxpdGllcyAoZS5nLiBTcGVhcm1hbiB2cy4gUGVhcnNvbiBjb3JyZWxhdGlvbnMpLiBIZXJlIHdlIHByZXNlbnQgYSB0YWJsZSBhbmQgYSB2aXN1YWxpemF0aW9uIHN1bW1hcml6aW5nIG91ciBmaW5kaW5ncy4gUmVmZXJlbmNlcyBtZW50aW9uZWQgaW4gdGhlIHRhYmxlIGJlbG93IGNhbiBiZSBmb3VuZCBbaGVyZV0oaHR0cHM6Ly9kb2NzLmdvb2dsZS5jb20vc3ByZWFkc2hlZXRzL2QvMWhjRUQ0X3JXU0dTRThUZDBhcVQwa1dxbExDQkF6Uk5LQlhqM3RTcmFtNTgvZWRpdD91c3A9c2hhcmluZykuICAKCmBgYHtyfQpsaXRfcmV2aWV3IDwtIHJlYWQuY3N2KCcvVXNlcnMvemV5bmVwZW5rYXZpL0Ryb3Bib3gvUG9sZHJhY2tMYWIvU1JPX1JldGVzdF9BbmFseXNlcy9pbnB1dC9saXRfcmV2aWV3X2ZpZ3VyZS5jc3YnKQoKbGl0X3JldmlldwpgYGAKCmBgYHtyIG1lc3NhZ2U9RkFMU0V9CmxpdF9yZXZpZXcgPSBsaXRfcmV2aWV3ICU+JQogIHNlcGFyYXRlKGR2LCBjKCJ0YXNrX2dyb3VwIiwgInZhciIpLCBzZXA9IlxcLiIscmVtb3ZlPUZBTFNFLGV4dHJhPSJtZXJnZSIpICU+JQogIG11dGF0ZSh0YXNrX2dyb3VwID0gZmFjdG9yKHRhc2tfZ3JvdXAsIGxldmVscyA9IHRhc2tfZ3JvdXBbb3JkZXIodGFzayldKSwKICAgICAgICAgdHlwZSA9IGFzLmNoYXJhY3Rlcih0eXBlKSkgJT4lCiAgbXV0YXRlKHRhc2tfZ3JvdXAgPSBnc3ViKCJfIiwgIiAiLCB0YXNrX2dyb3VwKSwKICAgICAgICAgdmFyID0gZ3N1YigiXyIsICIgIiwgdmFyKSkgJT4lCiAgYXJyYW5nZSh0YXNrX2dyb3VwLCByYXdfZml0LCB2YXIpICU+JQogIG11dGF0ZSh0YXNrX2dyb3VwID0gaWZlbHNlKHRhc2tfZ3JvdXAgPT0gInBzeWNob2xvZ2ljYWwgcmVmcmFjdG9yeSBwZXJpb2QgdHdvIGNob2ljZXMiLCAicHN5Y2hvbG9naWNhbCByZWZyYWN0b3J5IHBlcmlvZCIsIGlmZWxzZSh0YXNrX2dyb3VwID09ICJhbmdsaW5nIHJpc2sgdGFzayBhbHdheXMgc3VubnkiLCAiYW5nbGluZyByaXNrIHRhc2siLHRhc2tfZ3JvdXApKSkgJT4lCiAgbXV0YXRlKHRhc2tfZ3JvdXAgPSBnc3ViKCJzdXJ2ZXkiLCAiIiwgdGFza19ncm91cCkpICU+JQogIHNlbGVjdCgtbWVhc3VyZV9kZXNjcmlwdGlvbikKCgp0bXAgPSBsaXRfcmV2aWV3W2R1cGxpY2F0ZWQobGl0X3JldmlldyRyZWZlcmVuY2UpPT1GQUxTRSxdCgpucm93KHRtcCkKc3VtKHRtcCRzYW1wbGVfc2l6ZSkKbnJvdyhsaXRfcmV2aWV3KQpgYGAKCmBgYHtyfQpybSh0bXApCmBgYAoKTWVhc3VyZSBsZXZlbCBwbG90CgpgYGB7ciB3YXJuaW5nPUZBTFNFLCBtZXNzYWdlPUZBTFNFfQpsaXRfcmV2aWV3ID0gbGl0X3JldmlldyAlPiUKICBzZWxlY3QoLXJlZmVyZW5jZSkKYGBgCgpgYGB7ciB3YXJuaW5nPSBGQUxTRSwgbWVzc2FnZSA9RkFMU0UsIGVjaG89RkFMU0V9CnAxX2xlZ2VuZCA9IGxpdF9yZXZpZXcgJT4lCiAgZmlsdGVyKHRhc2sgPT0gJ3Rhc2snKSAlPiUKZ2dwbG90KGFlcyh5ID0gdmFyLCB4ID0gcmV0ZXN0X3JlbGlhYmlsaXR5KSkgKyAKICBnZW9tX3BvaW50KGFlcyhzaXplPXNhbXBsZV9zaXplLCBzaGFwZT10eXBlKSkrCiAgZmFjZXRfZ3JpZCh0YXNrX2dyb3Vwfi4sIHN3aXRjaCA9ICJ5Iiwgc2NhbGVzID0gImZyZWVfeSIsIHNwYWNlID0gImZyZWVfeSIpICsKICB0aGVtZShwYW5lbC5zcGFjaW5nID0gdW5pdCgwLjUsICJsaW5lcyIpLCAKICAgICAgICBzdHJpcC5wbGFjZW1lbnQgPSAib3V0c2lkZSIsCiAgICAgICAgc3RyaXAudGV4dC55ID0gZWxlbWVudF90ZXh0KGFuZ2xlPTE4MCksCiAgICAgICAgcGFuZWwuYmFja2dyb3VuZCA9IGVsZW1lbnRfcmVjdChmaWxsID0gTkEpLAogICAgICAgIHBhbmVsLmdyaWQubWFqb3IgPSBlbGVtZW50X2xpbmUoY29sb3VyID0gImdyZXk4MCIpLAogICAgICAgIGxlZ2VuZC5wb3NpdGlvbiA9ICdib3R0b20nKSArIAogIHhsYWIoIiIpKwogIHlsYWIoIiIpKwogIHNjYWxlX3hfY29udGludW91cyhsaW1pdHMgPSBjKC0wLjI1LDEpLCBicmVha3M9YygtMC4yNSwgMCwgMC4yNSwgMC41LCAwLjc1LCAxKSkrCiAgc2NhbGVfc2hhcGVfbWFudWFsKGJyZWFrcyA9IHNvcnQobGl0X3JldmlldyR0eXBlKSwgdmFsdWVzID0gYygxNSwgMTYsIDE3LCAzKSkKCnAxID0gbGl0X3JldmlldyAlPiUKICBmaWx0ZXIodGFzayA9PSAndGFzaycpICU+JQpnZ3Bsb3QoYWVzKHkgPSB2YXIsIHggPSByZXRlc3RfcmVsaWFiaWxpdHkpKSArIAogIGdlb21fcG9pbnQoYWVzKHNpemU9c2FtcGxlX3NpemUsIHNoYXBlID0gdHlwZSksIGNvbG9yPScjMDBCRkM0JykrCiAgZmFjZXRfZ3JpZCh0YXNrX2dyb3Vwfi4sIHN3aXRjaCA9ICJ5Iiwgc2NhbGVzID0gImZyZWVfeSIsIHNwYWNlID0gImZyZWVfeSIpICsKICB0aGVtZShwYW5lbC5zcGFjaW5nID0gdW5pdCgwLjUsICJsaW5lcyIpLCAKICAgICAgICBzdHJpcC5wbGFjZW1lbnQgPSAib3V0c2lkZSIsCiAgICAgICAgc3RyaXAudGV4dC55ID0gZWxlbWVudF90ZXh0KGFuZ2xlPTE4MCksCiAgICAgICAgcGFuZWwuYmFja2dyb3VuZCA9IGVsZW1lbnRfcmVjdChmaWxsID0gTkEpLAogICAgICAgIHBhbmVsLmdyaWQubWFqb3IgPSBlbGVtZW50X2xpbmUoY29sb3VyID0gImdyZXk4MCIpLAogICAgICAgIGxlZ2VuZC5wb3NpdGlvbiA9ICdib3R0b20nKSArIAogIHhsYWIoIiIpKwogIHlsYWIoIiIpKwogIHNjYWxlX3hfY29udGludW91cyhsaW1pdHMgPSBjKC0wLjI1LDEpLCBicmVha3M9YygtMC4yNSwgMCwgMC4yNSwgMC41LCAwLjc1LCAxKSkrCiAgc2NhbGVfc2hhcGVfbWFudWFsKGJyZWFrcyA9IHNvcnQobGl0X3JldmlldyR0eXBlKSwgdmFsdWVzID0gYygxNSwgMTYsIDE3LCAzKSkKCnAyID0gbGl0X3JldmlldyAlPiUKICBmaWx0ZXIodGFzayA9PSAnc3VydmV5JykgJT4lCmdncGxvdChhZXMoeSA9IHZhciwgeCA9IHJldGVzdF9yZWxpYWJpbGl0eSkpICsgCiAgZ2VvbV9wb2ludChhZXMoc2l6ZT1zYW1wbGVfc2l6ZSwgc2hhcGUgPSB0eXBlKSwgY29sb3IgPSAnI0Y4NzY2RCcpKwogIGZhY2V0X2dyaWQodGFza19ncm91cH4uLCBzd2l0Y2ggPSAieSIsIHNjYWxlcyA9ICJmcmVlX3kiLCBzcGFjZSA9ICJmcmVlX3kiKSArCiAgdGhlbWUocGFuZWwuc3BhY2luZyA9IHVuaXQoMC41LCAibGluZXMiKSwgCiAgICAgICAgc3RyaXAucGxhY2VtZW50ID0gIm91dHNpZGUiLAogICAgICAgIHN0cmlwLnRleHQueSA9IGVsZW1lbnRfdGV4dChhbmdsZT0xODApLAogICAgICAgIHBhbmVsLmJhY2tncm91bmQgPSBlbGVtZW50X3JlY3QoZmlsbCA9IE5BKSwKICAgICAgICBwYW5lbC5ncmlkLm1ham9yID0gZWxlbWVudF9saW5lKGNvbG91ciA9ICJncmV5ODAiKSkgKyAKICB4bGFiKCIiKSsKICB5bGFiKCIiKSsKICBzY2FsZV94X2NvbnRpbnVvdXMobGltaXRzID0gYygtMC4yNSwxKSwgYnJlYWtzPWMoLTAuMjUsIDAsIDAuMjUsIDAuNSwgMC43NSwgMSkpKwogIHNjYWxlX3NoYXBlX21hbnVhbChicmVha3MgPSBzb3J0KGxpdF9yZXZpZXckdHlwZSksIHZhbHVlcyA9IGMoMTUsIDE3LCAzKSkKCm15bGVnZW5kPC1nX2xlZ2VuZChwMV9sZWdlbmQpCgpwMyA8LSBhcnJhbmdlR3JvYihhcnJhbmdlR3JvYihwMSArdGhlbWUobGVnZW5kLnBvc2l0aW9uPSJub25lIiksCiAgICAgICAgICAgICAgICAgICAgICAgICBwMiArIHRoZW1lKGxlZ2VuZC5wb3NpdGlvbj0ibm9uZSIpLAogICAgICAgICAgICAgICAgICAgICAgICAgbnJvdz0xKSwKICAgICAgICAgICAgIG15bGVnZW5kLCBucm93PTIsaGVpZ2h0cz1jKDEwLCAxKSkKCmdnc2F2ZSgnTGl0X1Jldmlld19QbG90LmpwZycsIHBsb3QgPSBwMywgZGV2aWNlID0gImpwZWciLCBwYXRoID0gIi9Vc2Vycy96ZXluZXBlbmthdmkvRHJvcGJveC9Qb2xkcmFja0xhYi9TUk9fUmV0ZXN0X0FuYWx5c2VzL291dHB1dC9maWd1cmVzLyIsIHdpZHRoID0gMjQsIGhlaWdodCA9IDIwLCB1bml0cyA9ICJpbiIsIGRwaT0xMDApCnJtKHAxLCBwMiwgcDMsIHAxX2xlZ2VuZCwgbXlsZWdlbmQpCmBgYAoKYGBge3IgZWNobz1GQUxTRSwgb3V0LndpZHRoPScxMDAlJ30Ka25pdHI6OmluY2x1ZGVfZ3JhcGhpY3MoJy9Vc2Vycy96ZXluZXBlbmthdmkvRHJvcGJveC9Qb2xkcmFja0xhYi9TUk9fUmV0ZXN0X0FuYWx5c2VzL291dHB1dC9maWd1cmVzL0xpdF9SZXZpZXdfUGxvdC5qcGcnKQpgYGAKCkJlY2F1c2UgdGhpcyBwbG90IGlzIGRpZmZpY3VsdCB0byBkaWdlc3Qgd2Ugc3VtbWFyaXplIGl0IG9uIGEgdGFzayBsZXZlbCB0byBnaXZlIGEgZ2VuZXJhbCBzZW5zZSBvZiB0aGUgbWFpbiB0YWtlYXdheXMuIFRoaXMgcGxvdCBuYXR1cmFsbHkgZGlzcmVnYXJkcyBtdWNoIG9mIHRoZSBmaW5lIGdyYWluZWQgaW5mb3JtYXRpb24uCgpgYGB7cn0KcDFfdF9sZWdlbmQgPC0gbGl0X3JldmlldyAlPiUKICBmaWx0ZXIodGFzayA9PSAndGFzaycpICU+JQpnZ3Bsb3QoYWVzKHkgPSBmYWN0b3IodGFza19ncm91cCwgbGV2ZWxzPXJldih1bmlxdWUodGFza19ncm91cCkpKSwgeCA9IHJldGVzdF9yZWxpYWJpbGl0eSkpICsgCiAgZ2VvbV9wb2ludChhZXMoc2l6ZT1zYW1wbGVfc2l6ZSwgc2hhcGUgPSB0eXBlKSwgY29sb3I9J2JsYWNrJykrCiAgdGhlbWUoYXhpcy50ZXh0LnkgPSBlbGVtZW50X3RleHQoc2l6ZT00MyksCiAgICAgICAgbGVnZW5kLnBvc2l0aW9uID0gJ2JvdHRvbScsCiAgICAgICAgYXhpcy50ZXh0LnggPSBlbGVtZW50X3RleHQoc2l6ZT0yMyksCiAgICAgICAgYXhpcy50aXRsZS54ID0gZWxlbWVudF90ZXh0KHNpemU9MzApLCAKICAgICAgICBsZWdlbmQudGV4dCA9IGVsZW1lbnRfdGV4dChzaXplPTIwKSwgCiAgICAgICAgbGVnZW5kLmtleS53aWR0aCA9IHVuaXQoMC43NSwgImluY2hlcyIpLCAKICAgICAgICBsZWdlbmQudGl0bGUgPSBlbGVtZW50X3RleHQoc2l6ZT0yOCksCiAgICAgICAgbGVnZW5kLnNwYWNpbmcueCA9IHVuaXQoMC41LCAiaW5jaGVzIikpICsgCiAgZ3VpZGVzKHNpemUgPSBndWlkZV9sZWdlbmQob3ZlcnJpZGUuYWVzID0gbGlzdChzaXplPWMoOSwxOCwyOCkpKSwKICAgICAgICAgc2hhcGUgPSBndWlkZV9sZWdlbmQob3ZlcnJpZGUuYWVzID0gbGlzdChzaXplPTE4KSkpKwogIHhsYWIoIlJldGVzdCBSZWxpYWJpbGl0eSIpKwogIHlsYWIoIiIpKwogIHNjYWxlX3hfY29udGludW91cyhsaW1pdHMgPSBjKC0wLjI1LDEpLCBicmVha3M9YygtMC4yNSwgMCwgMC4yNSwgMC41LCAwLjc1LCAxKSkrCiAgc2NhbGVfeV9kaXNjcmV0ZShsYWJlbHMgPSBmdW5jdGlvbih4KSBzdHJfd3JhcCh4LCB3aWR0aCA9IDEwKSkrCiAgc2NhbGVfc2hhcGVfbWFudWFsKGJyZWFrcyA9IHNvcnQobGl0X3JldmlldyR0eXBlKSwgdmFsdWVzID0gYygxNSwgMTYsIDE3LCAzKSwgbmFtZT0iVHlwZSIpKwogIHNjYWxlX3NpemVfY29udGludW91cyhuYW1lID0gIlNhbXBsZSBTaXplIikrCiAgZ2VvbV92bGluZSh4aW50ZXJjZXB0ID0gMCwgY29sb3IgPSAicmVkIiwgc2l6ZSA9IDEpCgpwMV90IDwtIGxpdF9yZXZpZXcgJT4lCiAgZmlsdGVyKHRhc2sgPT0gJ3Rhc2snKSAlPiUKZ2dwbG90KGFlcyh5ID0gZmFjdG9yKHRhc2tfZ3JvdXAsIGxldmVscz1yZXYodW5pcXVlKHRhc2tfZ3JvdXApKSksIHggPSByZXRlc3RfcmVsaWFiaWxpdHkpKSArIAogIGdlb21fcG9pbnQoYWVzKHNpemU9c2FtcGxlX3NpemUsIHNoYXBlID0gdHlwZSksIGNvbG9yPScjMDBCRkM0JykrCiAgdGhlbWUoYXhpcy50ZXh0LnkgPSBlbGVtZW50X3RleHQoc2l6ZT00MyksCiAgICAgICAgbGVnZW5kLnBvc2l0aW9uID0gJ25vbmUnLAogICAgICAgIGF4aXMudGV4dC54ID0gZWxlbWVudF90ZXh0KHNpemU9MjMpLAogICAgICAgIGF4aXMudGl0bGUueCA9IGVsZW1lbnRfdGV4dChzaXplPTMwKSkgKyAKICB4bGFiKCJSZXRlc3QgUmVsaWFiaWxpdHkiKSsKICB5bGFiKCIiKSsKICBzY2FsZV94X2NvbnRpbnVvdXMobGltaXRzID0gYygtMC4yNSwxKSwgYnJlYWtzPWMoLTAuMjUsIDAsIDAuMjUsIDAuNSwgMC43NSwgMSkpKwogIHNjYWxlX3lfZGlzY3JldGUobGFiZWxzID0gZnVuY3Rpb24oeCkgc3RyX3dyYXAoeCwgd2lkdGggPSAxMCkpKwogIHNjYWxlX3NoYXBlX21hbnVhbChicmVha3MgPSBzb3J0KGxpdF9yZXZpZXckdHlwZSksIHZhbHVlcyA9IGMoMTUsIDE2LCAxNywgMykpKwogIHNjYWxlX3NpemVfY29udGludW91cyhyYW5nZSA9IGMoNSwgMzUpKSsKICBnZW9tX3ZsaW5lKHhpbnRlcmNlcHQgPSAwLCBjb2xvciA9ICJyZWQiLCBzaXplID0gMSkKCnAyX3QgPC0gbGl0X3JldmlldyAlPiUKICBmaWx0ZXIodGFzayA9PSAnc3VydmV5JykgJT4lCmdncGxvdChhZXMoeSA9IGZhY3Rvcih0YXNrX2dyb3VwLCBsZXZlbHM9cmV2KHVuaXF1ZSh0YXNrX2dyb3VwKSkpLCB4ID0gcmV0ZXN0X3JlbGlhYmlsaXR5KSkgKyAKICBnZW9tX3BvaW50KGFlcyhzaXplPXNhbXBsZV9zaXplLCBzaGFwZSA9IHR5cGUpLCBjb2xvcj0nI0Y4NzY2RCcpKwogIHRoZW1lKGF4aXMudGV4dC55ID0gZWxlbWVudF90ZXh0KHNpemU9NDMpLAogICAgICAgIGxlZ2VuZC5wb3NpdGlvbiA9ICdub25lJywKICAgICAgICBheGlzLnRleHQueCA9IGVsZW1lbnRfdGV4dChzaXplPTIzKSwKICAgICAgICBheGlzLnRpdGxlLnggPSBlbGVtZW50X3RleHQoc2l6ZT0zMCkpICsgCiAgeGxhYigiUmV0ZXN0IFJlbGlhYmlsaXR5IikrCiAgeWxhYigiIikrCiAgc2NhbGVfeF9jb250aW51b3VzKGxpbWl0cyA9IGMoLTAuMjUsMSksIGJyZWFrcz1jKC0wLjI1LCAwLCAwLjI1LCAwLjUsIDAuNzUsIDEpKSsKICBzY2FsZV95X2Rpc2NyZXRlKGxhYmVscyA9IGZ1bmN0aW9uKHgpIHN0cl93cmFwKHgsIHdpZHRoID0gMTApKSsKICBzY2FsZV9zaGFwZV9tYW51YWwoYnJlYWtzID0gc29ydChsaXRfcmV2aWV3JHR5cGUpLCB2YWx1ZXMgPSBjKDE1LCAxNiwgMTcsIDMpKSsKICBzY2FsZV9zaXplX2NvbnRpbnVvdXMocmFuZ2UgPSBjKDUsIDM1KSkrCiAgZ2VvbV92bGluZSh4aW50ZXJjZXB0ID0gMCwgY29sb3IgPSAicmVkIiwgc2l6ZSA9IDEpCgpteWxlZ2VuZDwtZ19sZWdlbmQocDFfdF9sZWdlbmQpCgpwM190IDwtIGFycmFuZ2VHcm9iKGFycmFuZ2VHcm9iKHAxX3QsIHAyX3QsIG5yb3c9MSksIG15bGVnZW5kLCBucm93PTIsaGVpZ2h0cz1jKDEwLCAxKSkKCmdnc2F2ZSgnTGl0X1Jldmlld19QbG90X3QuanBnJywgcGxvdCA9IHAzX3QsIGRldmljZSA9ICJqcGVnIiwgcGF0aCA9ICIvVXNlcnMvemV5bmVwZW5rYXZpL0Ryb3Bib3gvUG9sZHJhY2tMYWIvU1JPX1JldGVzdF9BbmFseXNlcy9vdXRwdXQvZmlndXJlcy8iLCB3aWR0aCA9IDI3LCBoZWlnaHQgPSA0OCwgdW5pdHMgPSAiaW4iLCBsaW1pdHNpemUgPSBGQUxTRSwgZHBpID0gNzIpCnJtKHAxX3QsIHAyX3QsIHAzX3QsIG15bGVnZW5kLCBwMV90X2xlZ2VuZCkKYGBgCgpgYGB7ciBlY2hvPUZBTFNFLCBvdXQud2lkdGg9JzEwMCUnfQprbml0cjo6aW5jbHVkZV9ncmFwaGljcygnL1VzZXJzL3pleW5lcGVua2F2aS9Ecm9wYm94L1BvbGRyYWNrTGFiL1NST19SZXRlc3RfQW5hbHlzZXMvb3V0cHV0L2ZpZ3VyZXMvTGl0X1Jldmlld19QbG90X3QuanBnJykKYGBgCgpBbiBpbnRlcmFjdGl2ZSB2ZXJzaW9uIG9mIHRoaXMgcGxvdCBjb3VsZCBiZSBmaW5kIFtoZXJlXShodHRwczovL3plbmthdmkuZ2l0aHViLmlvL1NST19SZXRlc3RfQW5hbHlzZXMvb3V0cHV0L3JlcG9ydHMvTGl0X1Jldmlld19GaWd1cmUuaHRtbCkKClRha2Vhd2F5cyBmcm9tIHRoaXMgcmV2aWV3IGFyZTogICAKLSBTdXJ2ZXkgbWVhc3VyZXMgaGF2ZSBiZWVuIHJlcG9ydGVkIHRvIGhpZ2hlciByZWxpYWJpbGl0eSBjb21wYXJlZCB0byB0YXNrIG1lYXN1cmVzICAKLSBTdXJ2ZXkgbWVhc3VyZXMgaGF2ZSBsZXNzIHZhcmlhYmlsaXR5IGluIHRoZSByZXBvcnRlZCByZWxpYWJpbHRpeSBlc3RpbWF0ZXMgY29tcGFyZWQgdG8gdGFzayBtZWFzdXJlcyAgIAoKIyMgTG9hZGluZyBkYXRhc2V0cwoKVGhlIHZhcmlhYmxlcyBpbmNsdWRlZCBpbiB0aGlzIHJlcG9ydCBhcmU6ICAKLSBtZWFuaW5nZnVsIHZhcmlhYmxlcyAoaW5jbHVkZXMgb25seSBoZGRkbSBwYXJhbWV0ZXJzKSAgCi0gRVogZGlmZnVzaW9uIHBhcmFtZXRlcnMgIAotIFJhdyBSVCBhbmQgQWNjdXJhY3kgbWVhc3VyZXMgIAotIFZhcmlhYmxlcyBmb3VuZCBpbiB0aGUgbGl0ZXJhdHVyZSAoZm9yIGNvbXBhcmlzb24pICAKCmBgYHtyIGVjaG89RkFMU0V9CiNHZXQgdmFyaWFibGVzIG9mIGludGVyZXN0IGZyb20gSWFuJ3MgcmVsZWFzZQp0bXAxIDwtIHJlYWQuY3N2KHBhc3RlMCh0ZXN0X2RhdGFfcGF0aCwnbWVhbmluZ2Z1bF92YXJpYWJsZXMuY3N2JykpCnRtcDIgPC0gcmVhZC5jc3YocGFzdGUwKHRlc3RfZGF0YV9wYXRoLCdtZWFuaW5nZnVsX3ZhcmlhYmxlc19ub0RETS5jc3YnKSkKdG1wMyA8LSByZWFkLmNzdihwYXN0ZTAodGVzdF9kYXRhX3BhdGgsJ21lYW5pbmdmdWxfdmFyaWFibGVzX0VaLmNzdicpKQpyZXRlc3RfcmVwb3J0X3ZhcnMgPSBjKG5hbWVzKHRtcDEpLCBuYW1lcyh0bXAyKSwgbmFtZXModG1wMykpCnJldGVzdF9yZXBvcnRfdmFycyA9IHVuaXF1ZShyZXRlc3RfcmVwb3J0X3ZhcnMpCmxpdF9yZXZfdmFycyA9IGFzLmNoYXJhY3Rlcih1bmlxdWUobGl0X3JldmlldyRkdilbd2hpY2godW5pcXVlKGxpdF9yZXZpZXckZHYpICVpbiUgcmV0ZXN0X3JlcG9ydF92YXJzID09IEZBTFNFKV0pCnJldGVzdF9yZXBvcnRfdmFycyA9IGMocmV0ZXN0X3JlcG9ydF92YXJzLCBsaXRfcmV2X3ZhcnMpCnJtKHRtcDEsIHRtcDIsIHRtcDMsIGxpdF9yZXZfdmFycykKYGBgCgojIyMgTG9hZCB0aW1lIDEgZGF0YQpgYGB7cn0KdGVzdF9kYXRhIDwtIHJlYWQuY3N2KHBhc3RlMChyZXRlc3RfZGF0YV9wYXRoLCd0MV9kYXRhL3ZhcmlhYmxlc19leGhhdXN0aXZlLmNzdicpKQoKdGVzdF9kYXRhIDwtIHRlc3RfZGF0YVssbmFtZXModGVzdF9kYXRhKSAlaW4lIHJldGVzdF9yZXBvcnRfdmFyc10KCnRlc3RfZGF0YSRYIDwtIGFzLmNoYXJhY3Rlcih0ZXN0X2RhdGEkWCkKbmFtZXModGVzdF9kYXRhKVt3aGljaChuYW1lcyh0ZXN0X2RhdGEpID09ICdYJyldIDwtJ3N1Yl9pZCcgCmBgYAoKRm9yIHJlZmVyZW5jZSBoZXJlIGFyZSB0aGUgdmFyaWFibGVzIHRoYXQgYXJlICoqbm90KiogaW5jbHVkZWQgaW4gdGhlIGFuYWx5c2VzIG9mIHRoZSByZW1haW5kZXIgb2YgdGhpcyByZXBvcnQgYmVjYXVzZSB0aGV5IHdlcmUgbm90IG9mIHRoZW9yZXRpY2FsIGludGVyZXN0IGluIGZhY3RvciBzdHJ1Y3R1cmUgYW5hbHlzZXMgb2YgdGhpcyBkYXRhIHNvIGZhci4gVGhlc2UgaW5jbHVkZSBkcmlmdCBkaWZmdXNpb24gYW5kIG90aGVyIG1vZGVsIHBhcmFtZXRlcnMgZm9yIHNwZWNpZmljIGNvbmRpdGlvbnMgd2l0aGluIGEgdGFzazsgc3VydmV5IHZhcmlhYmxlcyB0aGF0IGFyZSBub3QgcGFydCBvZiB0aGUgZGVwZW5kYW50IHZhcmlhYmxlcyBmb3IgdGhhdCBzdXJ2ZXkgaW4gdGhlIGxpdGVyYXR1cmUgYW5kIGRlbW9ncmFwaGljcyAodGhlc2UgYXJlIHNhdmVkIGZvciBwcmVkaWN0aW9uIGFuYWx5c2VzKS4KCmBgYHtyfQp0ZXN0X2RhdGEyIDwtIHJlYWQuY3N2KHBhc3RlMChyZXRlc3RfZGF0YV9wYXRoLCAndDFfZGF0YS92YXJpYWJsZXNfZXhoYXVzdGl2ZS5jc3YnKSkKCmRmIDwtIGRhdGEuZnJhbWUobmFtZXModGVzdF9kYXRhMilbd2hpY2gobmFtZXModGVzdF9kYXRhMikgJWluJSBuYW1lcyh0ZXN0X2RhdGEpID09IEZBTFNFKV0pCm5hbWVzKGRmKSA9IGMoJ3ZhcnMnKQoKZGYKYGBgCgpgYGB7ciBlY2hvPUZBTFNFfQpybSh0ZXN0X2RhdGEyLCBkZikKYGBgCgojIyMgTG9hZCB0aW1lIDIgZGF0YSAKYGBge3J9CnJldGVzdF9kYXRhIDwtIHJlYWQuY3N2KHBhc3RlMChyZXRlc3RfZGF0YV9wYXRoLCd2YXJpYWJsZXNfZXhoYXVzdGl2ZS5jc3YnKSkKCnJldGVzdF9kYXRhIDwtIHJldGVzdF9kYXRhWyxuYW1lcyhyZXRlc3RfZGF0YSkgJWluJSByZXRlc3RfcmVwb3J0X3ZhcnNdCgpyZXRlc3RfZGF0YSRYIDwtIGFzLmNoYXJhY3RlcihyZXRlc3RfZGF0YSRYKQpuYW1lcyhyZXRlc3RfZGF0YSlbd2hpY2gobmFtZXMocmV0ZXN0X2RhdGEpID09ICdYJyldIDwtJ3N1Yl9pZCcgCnJldGVzdF9kYXRhID0gcmV0ZXN0X2RhdGFbcmV0ZXN0X2RhdGEkc3ViX2lkICVpbiUgdGVzdF9kYXRhJHN1Yl9pZCxdCmBgYAoKIyMjIFJlcGxhY2UgSERETSBwYXJhbWV0ZXJzIGluIHQxIGRhdGEKClNpbmNlIEhERE0gcGFyYW1ldGVycyBkZXBlbmQgb24gdGhlIHNhbXBsZSBvbiB3aGljaCB0aGV5IGFyZSBmaXQgd2UgcmVmaXQgdGhlIG1vZGVsIG9uIHQxIGRhdGEgZm9yIHRoZSBzdWJqZWN0cyB0aGF0IGhhdmUgdDIgZGF0YS4gSGVyZSB3ZSByZXBsYWNlIHRoZSBIRERNIHBhcmFtZXRlcnMgaW4gdGhlIGN1cnJlbnQgdDEgZGF0YXNldCB3aXRoIHRoZXNlIHJlZml0dGVkIHZhbHVlcy4gCgpgYGB7cn0KaGRkbV9yZWZpdHMgPC0gcmVhZC5jc3YocGFzdGUwKHJldGVzdF9kYXRhX3BhdGgsJ3QxX2RhdGEvaGRkbV9yZWZpdHNfZXhoYXVzdGl2ZS5jc3YnKSkKCmhkZG1fcmVmaXRzID0gaGRkbV9yZWZpdHNbLG5hbWVzKGhkZG1fcmVmaXRzKSAlaW4lIHJldGVzdF9yZXBvcnRfdmFyc10KCmhkZG1fcmVmaXRzJFggPC0gYXMuY2hhcmFjdGVyKGhkZG1fcmVmaXRzJFgpCm5hbWVzKGhkZG1fcmVmaXRzKVt3aGljaChuYW1lcyhoZGRtX3JlZml0cykgPT0gJ1gnKV0gPC0nc3ViX2lkJyAKCiNGb3IgbGF0ZXIgY29tcGFyaXNvbiBvZiB3aGV0aGVyIGZpdHRpbmcgdGhlIERETSBwYXJhbWV0ZXJzIG9uIGZ1bGwgb3IgcmV0ZXN0IHNhbXBsZSBtYWtlcyBhIGJpZyBkaWZmZXJlbmNlCnRlc3RfZGF0YV9mdWxsX3NhbXBsZV9oZGRtIDwtIHRlc3RfZGF0YQoKI2Ryb3AgaGRkbSBjb2x1bW5zIGZyb20gdGVzdF9kYXRhCnRlc3RfZGF0YSA9IGNiaW5kKHRlc3RfZGF0YSRzdWJfaWQsIHRlc3RfZGF0YVssbmFtZXModGVzdF9kYXRhKSAlaW4lIG5hbWVzKGhkZG1fcmVmaXRzKSA9PSBGQUxTRV0pCgojZml4IG5hbWluZyBiZWZvcmUgbWVyZ2luZwpuYW1lcyh0ZXN0X2RhdGEpW3doaWNoKG5hbWVzKHRlc3RfZGF0YSkgPT0gJ3Rlc3RfZGF0YSRzdWJfaWQnKV0gPC0nc3ViX2lkJwoKI21lcmdlIGhkZG0gcmVmaXRzIHRvIHRlc3QgZGF0YQp0ZXN0X2RhdGEgPSBtZXJnZSh0ZXN0X2RhdGEsIGhkZG1fcmVmaXRzLCBieT0ic3ViX2lkIikKYGBgCgojIFJlc3VsdHMKCiMjIERhdGEgcXVhbGl0eSBjaGVja3MKCiMjIyBEZW1vZ3JhcGhpY3MgcmVsaWFiaWxpdHkKClBvaW50IGVzdGltYXRlcyBvZiByZWxpYWJpbGl0eSBmb3IgdGhlIGRlbW9ncmFwaGljIHZhcmlhYmVscy4KCmBgYHtyfQpudW1lcmljX2NvbHMgPSBjKCkKCmZvcihpIGluIDE6bGVuZ3RoKG5hbWVzKHRlc3RfZGVtb2cpKSl7CiAgaWYoaXMubnVtZXJpYyh0ZXN0X2RlbW9nWyxpXSkpewogICAgbnVtZXJpY19jb2xzIDwtIGMobnVtZXJpY19jb2xzLCBuYW1lcyh0ZXN0X2RlbW9nKVtpXSkKICB9Cn0KCmRlbW9nX3JlbF9kZiA8LSBkYXRhLmZyYW1lKHNwZWFybWFuID0gcmVwKE5BLCBsZW5ndGgobnVtZXJpY19jb2xzKSksCiAgICAgICAgICAgICAgICAgICAgIGljYyA9IHJlcChOQSwgbGVuZ3RoKG51bWVyaWNfY29scykpLAogICAgICAgICAgICAgICAgICAgICBwZWFyc29uID0gcmVwKE5BLCBsZW5ndGgobnVtZXJpY19jb2xzKSkpCgpyb3cubmFtZXMoZGVtb2dfcmVsX2RmKSA8LSBudW1lcmljX2NvbHMKCmZvcihpIGluIDE6bGVuZ3RoKG51bWVyaWNfY29scykpewogIGRlbW9nX3JlbF9kZltudW1lcmljX2NvbHNbaV0sICdzcGVhcm1hbiddIDwtIGdldF9zcGVhcm1hbihudW1lcmljX2NvbHNbaV0sIHQxX2RmID0gdGVzdF9kZW1vZywgdDJfZGYgPSByZXRlc3RfZGVtb2cpIAogIGRlbW9nX3JlbF9kZltudW1lcmljX2NvbHNbaV0sICdpY2MnXSA8LSBnZXRfaWNjKG51bWVyaWNfY29sc1tpXSwgdDFfZGYgPSB0ZXN0X2RlbW9nLCB0Ml9kZiA9IHJldGVzdF9kZW1vZykKICBkZW1vZ19yZWxfZGZbbnVtZXJpY19jb2xzW2ldLCAncGVhcnNvbiddIDwtIGdldF9wZWFyc29uKG51bWVyaWNfY29sc1tpXSwgdDFfZGYgPSB0ZXN0X2RlbW9nLCB0Ml9kZiA9IHJldGVzdF9kZW1vZykKfQoKZGVtb2dfcmVsX2RmID0gZGVtb2dfcmVsX2RmICU+JQogIG11dGF0ZSh2YXIgPSByb3cubmFtZXMoLikpICU+JQogIHNlbGVjdCh2YXIsIGljYywgc3BlYXJtYW4sIHBlYXJzb24pICU+JQogIGFycmFuZ2UoLWljYykKCmRlbW9nX3JlbF9kZgpgYGAKCmBgYHtyfQpzanQuZGYoZGVtb2dfcmVsX2RmICU+JSBtdXRhdGVfaWYoaXMubnVtZXJpYywgZnVucyhyb3VuZCguLCAzKSkpLCBkZXNjcmliZT1GLCBoaWRlLnByb2dyZXNzID0gVFJVRSwgc2hvdy5yb3duYW1lcyA9IEZBTFNFLCBmaWxlID0gIi9Vc2Vycy96ZXluZXBlbmthdmkvRHJvcGJveC9Qb2xkcmFja0xhYi9TUk9fUmV0ZXN0X0FuYWx5c2VzL291dHB1dC90YWJsZXMvZGVtb2dfcmVsX3RhYmxlLmRvYyIpCmBgYAoKCmBgYHtyfQpzdW1tYXJ5KGRlbW9nX3JlbF9kZikKYGBgCgpgYGB7ciBlY2hvPUZBTFNFfQpybSh0ZXN0X2RlbW9nLCByZXRlc3RfZGVtb2csIGRlbW9nX3JlbF9kZiwgbnVtZXJpY19jb2xzKQpgYGAKCiMjIyBFZmZlY3Qgb2YgZGVsYXlzIGJldHdlZW4gdGhlIHR3byBtZWFzdXJlbWVudHMKCkR1ZSB0byBvdXIgZGF0YSBjb2xsZWN0aW9uIHN0cmF0ZWd5IHdlIGRpZCBub3Qgc3RyaWN0bHkgY29udHJvbCBmb3IgdGhlIGRlbGF5IGJldHdlZW4gdGhlIHR3byBtZWFzdXJlbWVudHMgYXMgc3RhbmRhcmQgcHN5Y2hvbWV0cmljcyBzdHVkaWVzIG1lYXN1cmluZyByZXRlc3QgcmVsaWFiaWxpdHkgbWlnaHQuIAoKQW4gaW5kaXZpZHVhbCBkaWZmZXJlbmNlIG1lYXN1cmUgd291bGQgcHJlZmVyYWJseSByZW1haW4gc3RhYmxlIHJlZ2FyZGxlc3Mgb2YgdGhlIGRlbGF5IGJldHdlZW4gbXVsdGlwbGUgbWVhc3VyZW1lbnRzLgoKU2luY2Ugd2Ugb25seSBoYWQgdHdvIG1lYXN1cmVtZW50cyB3ZSBjb3VsZCBub3QgdGVzdCBkaXJlY3RseSB3aGV0aGVyIGEgbWVhc3VyZSBiZWNvbWVzIGxlc3MgcmVsaWFibGUgZGVwZW5kaW5nIG9uIHRoZSBkZWxheSBiZXR3ZWVuIHRoZSB0d28gdGltZSBwb2ludHMgKFRoZSBhdmVyYWdlIG51bWJlciBvZiBkYXlzIGJldHdlZW4gdHdvIG1lYXN1cmVtZW50cyB3b3VsZCBiZSB0aGUgc2FtZSBmb3IgYWxsIG1lYXN1cmVzIHNpbmNlIHJlbGlhYmlsaXR5IGlzIGEgbWVhc3VyZSBsZXZlbCBtZXRyaWMgd2hpbGUgZGF5cyBiZXR3ZWVuIGNvbXBsZXRpb24gYSBzdWJqZWN0IGxldmVsIG9uZSkuCgpTbyBpZiB5b3UgcmVncmVzcyB0aGUgYXZlcmFnZSBkaWZmZXJlbmNlIGZvciBlYWNoIG1lYXN1cmUgb24gYXZlcmFnZSBkZWxheSBiZXR3ZWVuIHR3byBtZWFzdXJlbWVudHMgeW91IGFyZSByZWdyZXNzaW5nIGEgdmVjdG9yIHdpdGggdmFyeWluZyBudW1iZXJzIG9uIGEgdmVjdG9yIG9mIHNhbWUgdmFsdWVzLCBsaWtlIGEgdCB0ZXN0IGFza2luZyBpZiB0aGUgbWVhbiBvZiB0aGUgdmFyeWluZyBjb2x1bW4sIGluIHRoaXMgY2FzZSB0aGUgYXZlcmFnZSBkaWZmZXJlbmNlIHNjb3JlLCBpcyBkaWZmZXJlbnQgdGhhbiB0aGUgdW5pcXVlIHZhbHVlIGluIHRoZSBzaW5nbGUgdmFsdWUgY29sdW1uLCBpLmUuIHRoZSBhdmVyYWdlIGRlbGF5IGJldHdlZW4gdHdvIHRpbWUgcG9pbnRzIChOb3RlIHRoYXQgdGhpcyBzaG91bGQgYmUgdHJ1ZSBpbiB0aGVvcnkgYnV0IGluIHByYWN0aWNlIHNpbmNlIGZvciBlYWNoIG1lYXN1cmUgdGhlcmUgbWlnaHQgc3ViamVjdHMgZm9yIHdob20gdGhlIGR2IGNvdWxkIG5vdCBiZSBjYWxjdWxhdGVkIHRoZXJlIG1pZ2h0IGJlID4xIHVuaXF1ZSB2YWx1ZXMgZm9yIHRoZSBhdmVyYWdlIGRlbGF5IGJldHdlZW4gdGhlIHR3byB0aW1lIHBvaW50cykuIFRoaXMgYW5hbHlzaXMgaXMgbm90IG1lYW5pbmdmdWwuIAoKSW5zdGVhZCBvZiB0aGUgc3VtbWFyeSBtZXRyaWMgbGlrZSB0aGUgcmV0ZXN0IHJlbGlhYmlsaXR5IGVzdGltYXRlIGZvciBlYWNoIG1lYXN1cmUgd2UgY2FuIGNoZWNrIHdoZXRoZXIgdGhlIGRpZmZlcmVuY2Ugc2NvcmUgZGlzdHJpYnV0aW9uIGZvciBlYWNoIG1lYXN1cmUgZGVwZW5kcyBvbiB0aGUgZGVsYXkgYmV0d2VlbiB0aGUgdHdvIG1lYXN1cmVtZW50cy4gU2luY2UgdGhlIGRpZmZlcmVuY2Ugc2NvcmUgZGlzdHJpYnV0aW9uIGlzIGF0IHN1YmplY3QgbGV2ZWwgd2UgY2FuIGNoZWNrIHdoZXRoZXIgdGhlIG9yZGVyIG9mIHN1YmplY3RzIGluIHRoaXMgZGlzdHJpYnV0aW9uIGRlcGVuZHMgb24gdGhlaXIgb3JkZXIgaW4gdGhlIGRpc3RyaWJ1dGlvbiBvZiBkYXlzIGJldHdlZW4gY29tcGxldGluZyB0aGUgdHdvIHRlc3RzLgoKTWFrZSBkYXRhIGZyYW1lIHdpdGggZGlmZmVyZW5jZSBiZXR3ZWVuIHR3byBzY29yZXMgZm9yIGVhY2ggbWVhc3VyZSBmb3IgZWFjaCBzdWJqZWN0LiBTaW5jZSB0aGUgc2NvcmVzIGZvciBkaWZmZXJlbnQgbWVhc3VyZXMgYXJlIG9uIGRpZmZlcmVudCBzY2FsZXMgZm9yIGNvbXBhcmFiaWxpdHkgdGhlIGRpZmZlcmVuY2Ugc2NvcmVzIGFyZSBub3JtYWxpemVkIChpZSBkZW1lYW5lZCBhbmQgZGl2aWRpZWQgYnkgdGhlIHNkIG9mIHRoZSBkaWZmZXJlbmNlIHNjb3JlIGRpc3RyaWJ1dGlvbi4pIE5vdGUgdGhhdCB0aGUgdmFyaWFuY2Ugb2YgdGhlIGRpZmZlcmVuY2Ugc2NvcmUgZGlzdHJpYnV0aW9uIGFjY291bnRzIGZvciB0aGUgdmFyaWFuY2UgaW4gYm90aCB0aW1lIHBvaW50cyBieSBzdW1taW5nIHRoZW0uIE5vcm1hbGl6YXRpb24gZXF1YXRlcyB0aGUgbWVhbnMgb2YgZWFjaCBkaWZmZXJlbmNlIHNjb3JlIGRpc3RyaWJ1dGlvbiB0byAwIHdoaWNoIHdvdWxkIG1hc2sgYW55IG1lYW5pbmdmdWwgY2hhbmdlIGJldHdlZW4gdGhlIHR3byB0aW1lIHBvaW50cyBidXQgdGhlIGFuYWx5c2lzIGhlcmUgZG9lcyBub3QgaW50ZXJwcmV0IHRoZSBtZWFuIG9mIHRoZSBkaWZmZXJlbmNlIHNjb3JlIGRpc3RyaWJ1dGlvbnMgYnV0IGlzIGludGVyZXN0ZWQgaW4gaXRzIHJlbGF0aW9uIHRvIHRoZSBkYXlzIGJldHdlZW4gY29tcGxldGlvbi4gV2UgY2hlY2sgaWYgdGhlIHZhcmlhYmxlcyBzaG93IHN5c3RlbWF0aWMgZGlmZmVyZW5jZXMgYmV0d2VlbiB0aGUgdHdvIHBvaW50cyBsYXRlci4KCkhlcmUgd2UgY2hlY2sgaWYgdGhlIGRpZmZlcmVuY2UgaXMgbGFyZ2VyIHRoZSBsb25nZXIgdGhlIGRlbGF5CgpgYGB7ciB3YXJuaW5nPUZBTFNFfQpudW1lcmljX2NvbHMgPSBjKCkKCmZvcihpIGluIDE6bGVuZ3RoKG5hbWVzKHRlc3RfZGF0YSkpKXsKICBpZihpcy5udW1lcmljKHRlc3RfZGF0YVssaV0pICYgbmFtZXModGVzdF9kYXRhKVtpXSAlaW4lIG5hbWVzKHJldGVzdF9kYXRhKSl7CiAgICBudW1lcmljX2NvbHMgPC0gYyhudW1lcmljX2NvbHMsIG5hbWVzKHRlc3RfZGF0YSlbaV0pCiAgfQp9Cgp0MV8yX2RpZmZlcmVuY2UgPSBkYXRhLmZyYW1lKCkKCmZvcihpIGluIDE6bGVuZ3RoKG51bWVyaWNfY29scykpewogIHRtcCA9IG1hdGNoX3QxX3QyKG51bWVyaWNfY29sc1tpXSxmb3JtYXQ9J3dpZGUnKQogIHRtcCA9IHRtcCAlPiUgCiAgbXV0YXRlKGRpZmZlcmVuY2UgPSBzY2FsZShgMmAgLSBgMWApKQogIHQxXzJfZGlmZmVyZW5jZSA9IHJiaW5kKHQxXzJfZGlmZmVyZW5jZSwgdG1wKQp9Cgp0MV8yX2RpZmZlcmVuY2UkZGlmZmVyZW5jZSA9IGFzLmRhdGEuZnJhbWUodDFfMl9kaWZmZXJlbmNlJGRpZmZlcmVuY2UpJFYxCgp0MV8yX2RpZmZlcmVuY2UgPSB0MV8yX2RpZmZlcmVuY2UgJT4lIHNlcGFyYXRlKGR2LCBjKCJ0YXNrIiwgImR2MiIpLCBzZXA9IlxcLiIsIHJlbW92ZT1GQUxTRSkKYGBgCgpgYGB7ciBlY2hvPUZBTFNFfQpybSh0bXAsIGkpCmBgYAoKQWRkIGNvbXBsZXRpb24gZGF0ZXMgdG8gdGhpcyBkYXRhIGZyYW1lLgoKYGBge3J9CnJldGVzdF90YXNrX2NvbXBfdGltZXMgPSByZWFkLmNzdihwYXN0ZTAocmV0ZXN0X2RhdGFfcGF0aCwgJ0xvY2FsL3JldGVzdF90YXNrX2NvbXBsZXRpb25fdGltZXMuY3N2JykpCnRlc3RfdGFza19jb21wX3RpbWVzID0gcmVhZC5jc3YocGFzdGUwKHJldGVzdF9kYXRhX3BhdGgsICdMb2NhbC90ZXN0X3Rhc2tfY29tcGxldGlvbl90aW1lcy5jc3YnKSkKdGFza19jb21wX3RpbWVzID0gbWVyZ2UocmV0ZXN0X3Rhc2tfY29tcF90aW1lcywgdGVzdF90YXNrX2NvbXBfdGltZXMsIGJ5PWMoJ3dvcmtlcl9pZCcsJ3Rhc2snKSkKcm0ocmV0ZXN0X3Rhc2tfY29tcF90aW1lcywgdGVzdF90YXNrX2NvbXBfdGltZXMpCnRhc2tfY29tcF90aW1lcyA9IHRhc2tfY29tcF90aW1lcyAlPiUKICBzZWxlY3QoLVgueCwgLVgueSkgJT4lCiAgbXV0YXRlKGZpbmlzaF9kYXkueCA9IGFzLkRhdGUoZmluaXNoX2RheS54KSwKICAgICAgICAgZmluaXNoX2RheS55ID0gYXMuRGF0ZShmaW5pc2hfZGF5LnkpLAogICAgICAgICBkYXlzX2J0dyA9IGZpbmlzaF9kYXkueC1maW5pc2hfZGF5LnkpICU+JQogIHJlbmFtZShzdWJfaWQ9d29ya2VyX2lkKQpgYGAKCmBgYHtyIHdhcm5pbmc9RkFMU0V9CnQxXzJfZGlmZmVyZW5jZSA9IG1lcmdlKHQxXzJfZGlmZmVyZW5jZSwgdGFza19jb21wX3RpbWVzWyxjKCdzdWJfaWQnLCAndGFzaycsJ2RheXNfYnR3JyldLCBieT1jKCdzdWJfaWQnLCAndGFzaycpKQpgYGAKCmBgYHtyIGVjaG89RkFMU0V9CnJtKHRhc2tfY29tcF90aW1lcykKYGBgCgpXaGF0IGRvZXMgdGhlIGRpc3RyaWJ1dGlvbiBvZiBkaWZmZXJlbmNlcyBsb29rIGxpa2U6IFRoZSBkaXN0cmlidXRpb24gb2YgZGlmZmVyZW5jZXMgYmV0d2VlbiB0d28gdGltZSBwb2ludHMgZm9yIGVhY2ggbWVhc3VyZSAKCmBgYHtyIHdhcm5pbmc9RkFMU0UsIG1lc3NhZ2U9RkFMU0V9CnQxXzJfZGlmZmVyZW5jZSAlPiUKICBnZ3Bsb3QoYWVzKGRpZmZlcmVuY2UsIGFscGhhPWR2KSkrCiAgZ2VvbV9oaXN0b2dyYW0ocG9zaXRpb249J2lkZW50aXR5JykrCiAgdGhlbWUobGVnZW5kLnBvc2l0aW9uID0gJ25vbmUnKQpgYGAKCkhvdyBkbyB0aGUgZGlmZmVyZW5jZSBzY29yZSBkaXN0cmlidXRpb25zIGxvb2sgbGlrZSB3aXRoIHJlc3BlY3QgdG8gdGhlIGRheXMgYmV0d2VlbiBjb21wbGV0aW9uPwoKYGBge3J9CnQxXzJfZGlmZmVyZW5jZSAlPiUKICBnZ3Bsb3QoKSsKICBnZW9tX3Ntb290aChhZXMoYXMubnVtZXJpYyhkYXlzX2J0dyksIGFicyhkaWZmZXJlbmNlKSwgZ3JvdXA9ZmFjdG9yKGR2KSksIG1ldGhvZD0nbG0nLCBzZT1GQUxTRSkrCiAgZ2VvbV9zbW9vdGgoYWVzKGFzLm51bWVyaWMoZGF5c19idHcpLCBhYnMoZGlmZmVyZW5jZSkpLCBtZXRob2Q9J2xtJywgY29sb3IgPSAiYmxhY2siLCBzZT1GQUxTRSkrCiAgdGhlbWUobGVnZW5kLnRpdGxlID0gZWxlbWVudF9ibGFuaygpKSsKICB4bGFiKCdEYXlzIGJldHdlZW4gY29tcGxldGlvbicpKwogIHlsYWIoJ0Fic29sdXRlIFNjYWxlZCBkaWZmZXJlbmNlIHNjb3JlJykKCmBgYAoKVG8gdGVzdCBpZiB0aGUgc2xvcGUgb2YgdGhlIGJsYWNrIGlzIHNpZ25pZmljYW50IHdlIHdvdWxkIHJ1biBhIG1peGVkIGVmZmVjdHMgbW9kZWwgd2l0aCBhIGZpeGVkIGVmZmVjdCBmb3IgZGF5cyBiZXR3ZWVuIGNvbXBsZXRpb24sIHJhbmRvbSBzbG9wZSBmb3IgZWFjaCBkdiBkZXBlbmRpbmcgb24gdGhlIGRheXMgYmV0d2VlbiBhbmQgcmFuZG9tIGludGVyY2VwdCBmb3IgZWFjaCBkdi4KCkJlZm9yZSBJIHdhcyB1c2luZyBzdWJqZWN0cyBhcyBhIHJhbmRvbSBlZmZlY3QgYnV0IGRheXMgYmV0d2VlbiB0aGUgdHdvIHRpbWUgcG9pbnRzIGZvciBlYWNoIG1lYXN1cmUgZGVwZW5kcyBvbiBzdWJqIGlkLiBXaGF0IHZhcmllcyByYW5kb21seSBpcyB3aGljaCBkdiB3ZSBhcmUgbG9va2luZyBmb3IgaXRzIGRpc3RyaWJ1dGlvbiBvZiBkaWZmZXJlbmNlcyBpbiByZWxhdGlvbiB0byB0aGUgZGF5cyBiZXR3ZWVuIHRoZSB0aW1lIHBvaW50cy4gU28gSSBjaGFuZ2VkIHRoZSBtb2RlbCB0byBoYXZlIGZpeGVkIGVmZmVjdCBmb3IgdGhlIGRheXMgYmV0d2VlbiwgYSByYW5kb20gc2xvcGUgZm9yIChkZXBlbmRlbnQgdmFyaWFibGVzIGNhbiBiZSBkaWZmZXJlbnRpYWxseSBzZW5zaXRpdmUgdG8gdGggZWZmZWN0IG9mIGRheXMgYmV0d2VlbikgYW5kIGEgcmFuZG9tIGludGVyY2VwdCBmb3IgZGVwZW5kZW50IHZhcmlhYmxlLgoKU2lnbmlmaWNhbnQgZml4ZWQgZWZmZWN0IHN1Z2dlc3RzIHRoYXQgb24gYXZlcmFnZSB0aGUgbG9uZ2VyIHRoZSBkZWxheSB0aGUgc21hbGxlciB0aGUgZGlmZmVyZW5jZS4KCmBgYHtyfQpzdW1tYXJ5KGxtZXJUZXN0OjpsbWVyKGFicyhkaWZmZXJlbmNlKSB+IHNjYWxlKGRheXNfYnR3KSsoc2NhbGUoZGF5c19idHcpIHwgZHYpLCBkYXRhPXQxXzJfZGlmZmVyZW5jZSkpIApgYGAKCkJ1dCBpZiBJIGp1c3QgcnVuIG9uZSBtaXhlZCBlZmZlY3RzIG1vZGVsIHRoZW4gd2UgZG9uJ3QgZ2V0IGEgc2Vuc2Ugb2YgdGhlIHNpbXBsZSBlZmZlY3RzIChob3cgbWFueSBvZiB0aGUgdmFyaWFibGVzIHRoaXMgZWZmZWN0IG9mIHRoZSBkYXlzIGJldHdlZW4gaXMgc2lnbmlmaWNhbnQgYW5kIGluIHdoaWNoIGRpcmVjdGlvbikuIEkgY2FuIHJ1biBpdCBzZXBhcmF0ZWx5IGZvciBlYWNoIGR2IHRvIHNlZSBpZiBhbGwgZGlmZmVyZW5jZSBzY29yZSBkaXN0cmlidXRpb25zIGFyZSBhZmZlY3RlZCB0aGUgc2FtZSB3YXkgZGVwZW5kaW5nIG9uIHRoZSBkYXlzIGJldHdlZW4gY29tcGxldGlvbi4KCmBgYHtyfQpnZXRfZGVsYXlfZWZmZWN0ID0gZnVuY3Rpb24oZGYpewogIG1vZCA9IGxtKGFicyhkaWZmZXJlbmNlKSB+IHNjYWxlKGRheXNfYnR3KSwgZGF0YSA9IGRmKQogIG91dCA9IGRhdGEuZnJhbWUoZXN0aW1hdGU9Y29lZihzdW1tYXJ5KG1vZCkpWyJzY2FsZShkYXlzX2J0dykiLCJFc3RpbWF0ZSJdLCBwdmFsPWNvZWYoc3VtbWFyeShtb2QpKVsic2NhbGUoZGF5c19idHcpIiwiUHIoPnx0fCkiXSkKICByZXR1cm4ob3V0KQp9CgpzaWdfZGF5c19lZmZlY3QgPSB0MV8yX2RpZmZlcmVuY2UgJT4lCiAgZ3JvdXBfYnkoZHYpICU+JQogIGRvKGdldF9kZWxheV9lZmZlY3QoLikpICU+JQogIGZpbHRlcihwdmFsPDAuMDUpCgpzaWdfZGF5c19lZmZlY3QKYGBgCgpGb3IgdmlzdWFsaXphdGlvbiBJIHVzZWQgdG8gc3VtbWFyaXplIHRoZSBkaWZmZXJlbmNlIHNjb3JlcyBwZXIgcGVyc29uIGJ5IGxvb2tpbmcgYXQgdGhlIGF2ZXJhZ2UgZGlmZmVyZW5jZSBwZXIgdGFzayBwZXIgc3ViamVjdCBhbmQgcGxvdCB0aGF0IGFnYWluc3QgdGhlIG51bWJlciBvZiBkYXlzIGJldHdlZW4gY29tcGxldGlvbi4gVGhpcyB5aWVsZHMgbXVsdGlwbGUgcG9pbnRzIGZvciBlYWNoIHZhbHVlIG9uIHggcmVwcmVzZW50aW5nIHRoZSBhdmVyYWdlIGRpZmZlcmVuY2UgcGVyIHRhc2sgZm9yIGVhY2ggc3ViamVjdC4gQnV0IHZhcmlhYmxlcyB3aXRoaW4gYSB0YXNrIGNvdWxkIGdvIGluIGRpZmZlcmVudCBkaXJlY3Rpb25zIChlLmcuIGlmIHBhdGllbnQgcHJvcG9ydGlvbiBpbmNyZWFzZXMgZGlzY291bnQgcmF0ZSBkZWNyZWFzZXMpIHNvIHRoaXMgZG9lc24ndCBzZWVtIGxpa2UgYSBnb29kIGlkZWEpIAoKSW5zdGVhZCBJIG5vdyBwbG90IGFsbCB0aGUgZGF0YSBncm91cGluZyBieSBkZXBlbmRhbnQgdmFyaWFibGUgYW5kIGNvbG9yaW5nIGRlcGVuZGluZyBvbiB3aGV0aGVyIHRoZSBkaWZmZXJlbmNlIHNjb3JlIGRpc3RyaWJ1dGlvbiBmb3IgdGhhdCB2YXJpYWJsZSBoYXMgYSBzaWduaWZpY2FudCBzbG9wZSB3aGVuIHJlZ3Jlc3NlZCBvdmVyIGRheXMgYmV0d2Vlbi4gVGhlIGJsYWNrIGlzIHRoZSBtYWluIGVmZmVjdCBmb3IgdGhlIGxhcmdlIG11bHRpbGV2ZWwgbW9kZWwgKHRoaXMgaXMgdGhlIHNhbWUgcGxvdCBhcyBhYm92ZSBjb2xvcmVkIGJ5IHdoZXRoZXIgdGhlIGRpZmZlcmVuY2Ugc2NvcmUgZGlzdHJpYnV0aW9uIGZvciBlYWNoIGR2IGhhcyBhIHNpZ25pZmljYW50IHNsb3BlIGRlcGVuZGluZyBvbiBkYXlzIGJldHdlZW4pLgoKYGBge3J9CnQxXzJfZGlmZmVyZW5jZSAlPiUKICBtdXRhdGUoc2lnX2RheXNfZWZmZWN0ID0gaWZlbHNlKGR2ICVpbiUgc2lnX2RheXNfZWZmZWN0JGR2LCAxLCAwKSklPiUKICBhcnJhbmdlKC1zaWdfZGF5c19lZmZlY3QpICU+JQogIGdncGxvdCgpKwogIHN0YXRfc21vb3RoIChhZXMoYXMubnVtZXJpYyhkYXlzX2J0dyksIGFicyhkaWZmZXJlbmNlKSwgCiAgICAgICAgICAgICAgICAgIGdyb3VwPWZhY3RvcihkdiwgbGV2ZWxzPXVuaXF1ZShkdltvcmRlcihzaWdfZGF5c19lZmZlY3QpXSksIG9yZGVyZWQ9VFJVRSksIAogICAgICAgICAgICAgICAgICBjb2xvcj1mYWN0b3Ioc2lnX2RheXNfZWZmZWN0LCBsZXZlbHMgPSBjKDEsMCkpKSxnZW9tPSJsaW5lIiwgYWxwaGE9MC41LCBtZXRob2Q9J2xtJykrCiAgZ2VvbV9zbW9vdGgoYWVzKGFzLm51bWVyaWMoZGF5c19idHcpLCBhYnMoZGlmZmVyZW5jZSkpLCBtZXRob2Q9J2xtJywgY29sb3IgPSAiYmxhY2siLCBzZT1GQUxTRSkrCiAgdGhlbWUobGVnZW5kLnRpdGxlID0gZWxlbWVudF9ibGFuaygpKSsKICB4bGFiKCdEYXlzIGJldHdlZW4gY29tcGxldGlvbicpKwogIHlsYWIoJ1NjYWxlZCBkaWZmZXJlbmNlIHNjb3JlJykrCiAgc2NhbGVfY29sb3JfZGlzY3JldGUoYnJlYWtzPWMoMCwxKSwKICAgICAgICAgICAgICAgICAgICAgICBsYWJlbHM9YygiTlMiLCAiU2lnIikpCgpnZ3NhdmUoJ0RheXNCdHdFZmZlY3QuanBnJywgZGV2aWNlID0gImpwZWciLCBwYXRoID0gIi9Vc2Vycy96ZXluZXBlbmthdmkvRHJvcGJveC9Qb2xkcmFja0xhYi9TUk9fUmV0ZXN0X0FuYWx5c2VzL291dHB1dC9maWd1cmVzLyIsIHdpZHRoID0gNiwgaGVpZ2h0ID0gNCwgdW5pdHMgPSAiaW4iLCBsaW1pdHNpemUgPSBGQUxTRSwgZHBpID0gMTAwKQpgYGAKCkNvbmNsdXNpb246IFRoZSBhdmVyYWdlIGRpZmZlcmVuY2UgYmV0d2VlbiB0aGUgc2NvcmVzIG9mIGEgbWVhc3VyZSBpZiBhbnl0aGluZyBkZWNyZWFzZXMgd2l0aCB0aGUgaW5jcmVhc2VkIGRlbGF5LiBUaGlzIGZpeGVkIGVmZmVjdCBtYXNrcyB0aGUgc2ltcGxlIGVmZmVjdCBmb3IgZWFjaCBkdi4gV2hlbiBsb29rZWQgYXQgaW5kaXZpZHVhbGx5IG9ubHkgMTAgdmFyaWFibGVzIHNob3cgYSBkZXBlbmRlbmNlIG9uIGRheXMgYmV0d2VlbiBjb21wbGV0aW9uLiBXaGlsZSA3IG9mIHRoZW0gc2hvdyBkZWNyZWFzaW5nIGRpZmZlcmVuY2VzIGRlcGVuZGluZyBvbiBkYXlzIGJldHdlZW4gMyBkbyBzaG93IGluY3JlYXNpbmcgZGlmZmVyZW5jZXMuClRoaXMgc3VnZ2VzdHMgdGhhdCB0aGUgbGFjayBvZiBzdHJpY3RlciBjb250cm9sIG92ZXIgdGhlIGRheXMgYmV0d2VlbiBjb21wbGV0aW9uIG9mIHRoZSBtZWFzdXJlbWVudCBkb2VzIG5vdCBoYXZlIGEgbGFyZ2UgZWZmZWN0IG9uIHRoZSBzdGFiaWxpdHkgb2YgbW9zdCBtZWFzdXJlcy4KCmVmZmVjdCBvZiB3aGV0aGVyIGRheXMgYmV0d2VlbiBsZWFkcyB0byBhIHNpZyBkaWZmZXJlbmNlIG9uIHJlbGlhYmlsaXR5IG9mIHRoYXQgdmFyaWFibGU/IHBsb3QgY29lZiBvZiBzaWdfZGF5X2VmZmVjdCBvdmVyIHJlbF9kZiBwb2ludCBlc3RpbWF0ZQoKRG8gdGhlIHZhcmlhYmxlcyB0aGF0IHNob3cgc2lnbmlmaWNhbnQgZGVwZW5kZW5jZSBvbiB0aGUgZGVsYXkgYmV0d2VlbiB0aGUgY29tcGxldGlvbiB0aW1lcyBoYXZlIHN5c3RlbWF0aWNhbGx5IGxvd2VyL2hpZ2hlciByZWxpYWJpbGl0eT8gTG9va2luZyBhdCB0aGUgcmVsaWFiaWxpdHkgcG9pbnQgZXN0aW1hdGUgb3ZlciB0aGUgc2l6ZSBvZiB0aGUgZGVsYXkgZWZmZWN0LiBUaGVyZSBhcmUgdmVyeSBmZXcgdmFyaWFibGVzIHRvIGNvbXBhcmUgYW55d2F5IGJ1dCByZWdhcmRsZXNzIGRvZXNuJ3QgbG9vayBjb25jbHVzaXZlL2NvbmNlcm5pbmcuCgpgYGB7cn0KI0NyZWF0ZSBkZiBvZiBwb2ludCBlc3RpbWF0ZSByZWxpYWJpbGl0aWVzCnJlbF9kZiA8LSBkYXRhLmZyYW1lKHNwZWFybWFuID0gcmVwKE5BLCBsZW5ndGgobnVtZXJpY19jb2xzKSksCiAgICAgICAgICAgICAgICAgICAgIGljYyA9IHJlcChOQSwgbGVuZ3RoKG51bWVyaWNfY29scykpLAogICAgICAgICAgICAgICAgICAgICBwZWFyc29uID0gcmVwKE5BLCBsZW5ndGgobnVtZXJpY19jb2xzKSksCiAgICAgICAgICAgICAgICAgICAgIHBhcnRpYWxfZXRhX3NxID0gcmVwKE5BLCBsZW5ndGgobnVtZXJpY19jb2xzKSksCiAgICAgICAgICAgICAgICAgICAgIHNlbSA9IHJlcChOQSwgbGVuZ3RoKG51bWVyaWNfY29scykpLAogICAgICAgICAgICAgICAgICAgICB2YXJfc3VicyA9IHJlcChOQSwgbGVuZ3RoKG51bWVyaWNfY29scykpLAogICAgICAgICAgICAgICAgICAgICB2YXJfaW5kID0gcmVwKE5BLCBsZW5ndGgobnVtZXJpY19jb2xzKSksCiAgICAgICAgICAgICAgICAgICAgIHZhcl9yZXNpZCA9IHJlcChOQSwgbGVuZ3RoKG51bWVyaWNfY29scykpKQoKcm93Lm5hbWVzKHJlbF9kZikgPC0gbnVtZXJpY19jb2xzCgpmb3IoaSBpbiAxOmxlbmd0aChudW1lcmljX2NvbHMpKXsKICByZWxfZGZbbnVtZXJpY19jb2xzW2ldLCAnc3BlYXJtYW4nXSA8LSBnZXRfc3BlYXJtYW4obnVtZXJpY19jb2xzW2ldKSAKICByZWxfZGZbbnVtZXJpY19jb2xzW2ldLCAnaWNjJ10gPC0gZ2V0X2ljYyhudW1lcmljX2NvbHNbaV0pCiAgcmVsX2RmW251bWVyaWNfY29sc1tpXSwgJ3BlYXJzb24nXSA8LSBnZXRfcGVhcnNvbihudW1lcmljX2NvbHNbaV0pCiAgcmVsX2RmW251bWVyaWNfY29sc1tpXSwgJ3BhcnRpYWxfZXRhX3NxJ10gPC0gZ2V0X3BhcnRpYWxfZXRhKG51bWVyaWNfY29sc1tpXSkKICByZWxfZGZbbnVtZXJpY19jb2xzW2ldLCAnc2VtJ10gPC0gZ2V0X3NlbShudW1lcmljX2NvbHNbaV0pCiAgcmVsX2RmW251bWVyaWNfY29sc1tpXSwgJ3Zhcl9zdWJzJ10gPC0gZ2V0X3Zhcl9icmVha2Rvd24obnVtZXJpY19jb2xzW2ldKSRzdWJzCiAgcmVsX2RmW251bWVyaWNfY29sc1tpXSwgJ3Zhcl9pbmQnXSA8LSBnZXRfdmFyX2JyZWFrZG93bihudW1lcmljX2NvbHNbaV0pJGluZAogIHJlbF9kZltudW1lcmljX2NvbHNbaV0sICd2YXJfcmVzaWQnXSA8LSBnZXRfdmFyX2JyZWFrZG93bihudW1lcmljX2NvbHNbaV0pJHJlc2lkCn0KCnJlbF9kZiRkdiA9IHJvdy5uYW1lcyhyZWxfZGYpCnJvdy5uYW1lcyhyZWxfZGYpID0gc2VxKDE6bnJvdyhyZWxfZGYpKQpyZWxfZGYkdGFzayA9ICd0YXNrJwpyZWxfZGZbZ3JlcCgnc3VydmV5JywgcmVsX2RmJGR2KSwgJ3Rhc2snXSA9ICdzdXJ2ZXknCnJlbF9kZltncmVwKCdob2x0JywgcmVsX2RmJGR2KSwgJ3Rhc2snXSA9ICJ0YXNrIgpyZWxfZGYgPSByZWxfZGYgJT4lCiAgc2VsZWN0KGR2LCB0YXNrLCBzcGVhcm1hbiwgaWNjLCBwZWFyc29uLCBwYXJ0aWFsX2V0YV9zcSwgc2VtLCB2YXJfc3VicywgdmFyX2luZCwgdmFyX3Jlc2lkKQojIHJvdy5uYW1lcyhyZWxfZGYpID0gTlVMTApgYGAKCmBgYHtyfQpzaWdfZGF5c19lZmZlY3QgJT4lCiAgbGVmdF9qb2luKHJlbF9kZiwgYnk9J2R2JykgJT4lCiAgZ2dwbG90KGFlcyhlc3RpbWF0ZSwgaWNjKSkrCiAgZ2VvbV9wb2ludCgpCmBgYAoKIyMjIE92ZXJsYXBwaW5nIHN1cnZlcnkgcXVlc3Rpb25zCgpTb21lIHN1cnZleXMgaGF2ZSBvdmVybGFwcGluZyBxdWVzdGlvbnMuIERvIHRoZXNlIGNvcnJlbGF0ZSB3aXRoaW4gYW5kIGFjcm9zcyBzZXNzaW9ucz8KCkZpcnN0IGRldGVybWluZSB0aGUgb3ZlcmxhcHBpbmcgcXVlc3Rpb25zLgoKYGBge3J9CnRtcCA9IHJlYWQuY3N2KGd6ZmlsZShwYXN0ZTAocmV0ZXN0X2RhdGFfcGF0aCwgJ2l0ZW1zLmNzdi5neicpKSkKdG1wID0gdG1wICU+JSAKICBmaWx0ZXIod29ya2VyID09ICdzMDA1JykgJT4lIAogIHNlbGVjdChpdGVtX0lELCBpdGVtX3RleHQpICU+JSAKICBtdXRhdGUoaXRlbV90ZXh0ID0gdHJpbXdzKGFzLmNoYXJhY3RlcihpdGVtX3RleHQpKSkgJT4lCiAgdW5pdGUoaXRlbSwgYygiaXRlbV9JRCIsICJpdGVtX3RleHQiKSwgc2VwID0gIl9fXyIpCgpjb21iID0gYXMuZGF0YS5mcmFtZSh0KGNvbWJuKHVuaXF1ZSh0bXAkaXRlbSksMikpKQoKZHVwbGljYXRlX2l0ZW1zID0gY29tYiAlPiUgCiAgZmlsdGVyKGdyZXBsKCdkb3NwZXJ0JywgVjEpPT1GQUxTRSkgJT4lCiAgZmlsdGVyKGdyZXBsKCdzZWxlY3Rpb25fb3B0aW1pemF0aW9uJywgVjEpPT1GQUxTRSkgJT4lCiAgZmlsdGVyKGdyZXBsKCdzZW5zYXRpb25fc2Vla2luZycsIFYxKT09RkFMU0UpICU+JQogIHNlcGFyYXRlKFYxLCBjKCJpdGVtMV9JRCIsICJpdGVtMV90ZXh0IiksIHNlcD0iX19fIikgJT4lCiAgc2VwYXJhdGUoVjIsIGMoIml0ZW0yX0lEIiwgIml0ZW0yX3RleHQiKSwgc2VwPSJfX18iKSAlPiUKICBtdXRhdGUoc2ltaWxhcml0eSA9IGxldmVuc2h0ZWluU2ltKGl0ZW0xX3RleHQsIGl0ZW0yX3RleHQpKSAlPiUKICBmaWx0ZXIoc2ltaWxhcml0eT4wLjgpICU+JQogIHNlbGVjdChpdGVtMV9JRCwgaXRlbTJfSUQsIGl0ZW0xX3RleHQsIGl0ZW0yX3RleHQpCgpkdXBsaWNhdGVfaXRlbXMKYGBgCgpgYGB7ciB3YXJuaW5nPUZBTFNFLCBtZXNzYWdlPUZBTFNFfQojc3VydmV5cyB0byByZWFkIGluCmV4dHJhY3RfaXRlbXMgPSBjKCd3b3JrZXInLHVuaXF1ZSh3aXRoKGR1cGxpY2F0ZV9pdGVtcywgYyhpdGVtMV9JRCwgaXRlbTJfSUQpKSkpCgojY29ycmVsYXRpb25zIHRvIGNvbXB1dGU6CiNpdGVtMV90MSAtIGl0ZW0yX3QxLCAKI2l0ZW0xX3QyIC0gaXRlbTJfdDIsIAojaXRlbTFfdDEgLSBpdGVtMl90MiwgCiNpdGVtMV90MiAtIGl0ZW0yX3QxCgpkdXBsaWNhdGVfaXRlbXNfZGF0YV90MSA9IHJlYWQuY3N2KHBhc3RlMCh0ZXN0X2RhdGFfcGF0aCwgJ3N1YmplY3RfeF9pdGVtcy5jc3YnKSkKZHVwbGljYXRlX2l0ZW1zX2RhdGFfdDIgPSByZWFkLmNzdihwYXN0ZTAocmV0ZXN0X2RhdGFfcGF0aCwgJ3N1YmplY3RfeF9pdGVtcy5jc3YnKSkKCmR1cGxpY2F0ZV9pdGVtc19kYXRhX3QxID0gZHVwbGljYXRlX2l0ZW1zX2RhdGFfdDEgJT4lCiAgZmlsdGVyKHdvcmtlciAlaW4lIGR1cGxpY2F0ZV9pdGVtc19kYXRhX3QyJHdvcmtlcikgJT4lCiAgc2VsZWN0KGV4dHJhY3RfaXRlbXMpCgpkdXBsaWNhdGVfaXRlbXNfZGF0YV90Mj1kdXBsaWNhdGVfaXRlbXNfZGF0YV90MiAlPiUKICBmaWx0ZXIod29ya2VyICVpbiUgZHVwbGljYXRlX2l0ZW1zX2RhdGFfdDEkd29ya2VyKSAlPiUKICBzZWxlY3QoZXh0cmFjdF9pdGVtcykKCmR1cGxpY2F0ZV9pdGVtcyA9IGR1cGxpY2F0ZV9pdGVtcyAlPiUKICBtdXRhdGUodDFfdDFfY29yID0gTkEsCiAgICAgICAgIHQyX3QyX2NvciA9IE5BLAogICAgICAgICB0MV90Ml9jb3IgPSBOQSwKICAgICAgICAgdDJfdDFfY29yID0gTkEsCiAgICAgICAgIHQxX3QxX3BvbHljb3IgPSBOQSwKICAgICAgICAgdDJfdDJfcG9seWNvciA9IE5BLAogICAgICAgICB0MV90Ml9wb2x5Y29yID0gTkEsCiAgICAgICAgIHQyX3QxX3BvbHljb3IgPSBOQSkKCmZvcihpIGluIDE6bnJvdyhkdXBsaWNhdGVfaXRlbXMpKXsKICBkdXBsaWNhdGVfaXRlbXMkdDFfdDFfY29yW2ldID0gYWJzKGNvcihkdXBsaWNhdGVfaXRlbXNfZGF0YV90MVssYyhkdXBsaWNhdGVfaXRlbXMkaXRlbTFfSURbaV0pXSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGR1cGxpY2F0ZV9pdGVtc19kYXRhX3QxWyxjKGR1cGxpY2F0ZV9pdGVtcyRpdGVtMl9JRFtpXSldKSkKICAKICBkdXBsaWNhdGVfaXRlbXMkdDJfdDJfY29yW2ldID0gYWJzKGNvcihkdXBsaWNhdGVfaXRlbXNfZGF0YV90MlssYyhkdXBsaWNhdGVfaXRlbXMkaXRlbTFfSURbaV0pXSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGR1cGxpY2F0ZV9pdGVtc19kYXRhX3QyWyxjKGR1cGxpY2F0ZV9pdGVtcyRpdGVtMl9JRFtpXSldKSkKICAKICBkdXBsaWNhdGVfaXRlbXMkdDFfdDJfY29yW2ldID0gYWJzKGNvcihkdXBsaWNhdGVfaXRlbXNfZGF0YV90MVssYyhkdXBsaWNhdGVfaXRlbXMkaXRlbTFfSURbaV0pXSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGR1cGxpY2F0ZV9pdGVtc19kYXRhX3QyWyxjKGR1cGxpY2F0ZV9pdGVtcyRpdGVtMl9JRFtpXSldKSkKICAKICBkdXBsaWNhdGVfaXRlbXMkdDJfdDFfY29yW2ldID0gYWJzKGNvcihkdXBsaWNhdGVfaXRlbXNfZGF0YV90MlssYyhkdXBsaWNhdGVfaXRlbXMkaXRlbTFfSURbaV0pXSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGR1cGxpY2F0ZV9pdGVtc19kYXRhX3QxWyxjKGR1cGxpY2F0ZV9pdGVtcyRpdGVtMl9JRFtpXSldKSkKICAKICBkdXBsaWNhdGVfaXRlbXMkdDFfdDFfcG9seWNvcltpXSA9IGFicyhwb2x5Y2hvcmljKGRhdGEuZnJhbWUoZHVwbGljYXRlX2l0ZW1zX2RhdGFfdDFbLGMoZHVwbGljYXRlX2l0ZW1zJGl0ZW0xX0lEW2ldKV0sCiAgICAgICAgICAgICAgICAgICAgICBkdXBsaWNhdGVfaXRlbXNfZGF0YV90MVssYyhkdXBsaWNhdGVfaXRlbXMkaXRlbTJfSURbaV0pXSkpJHJob1syXSkKCiAgZHVwbGljYXRlX2l0ZW1zJHQyX3QyX3BvbHljb3JbaV0gPSBhYnMocG9seWNob3JpYyhkYXRhLmZyYW1lKGR1cGxpY2F0ZV9pdGVtc19kYXRhX3QyWyxjKGR1cGxpY2F0ZV9pdGVtcyRpdGVtMV9JRFtpXSldLAogICAgICAgICAgICAgICAgICAgICAgZHVwbGljYXRlX2l0ZW1zX2RhdGFfdDJbLGMoZHVwbGljYXRlX2l0ZW1zJGl0ZW0yX0lEW2ldKV0pKSRyaG9bMl0pCgogIGR1cGxpY2F0ZV9pdGVtcyR0MV90Ml9wb2x5Y29yW2ldID0gYWJzKHBvbHljaG9yaWMoZGF0YS5mcmFtZShkdXBsaWNhdGVfaXRlbXNfZGF0YV90MVssYyhkdXBsaWNhdGVfaXRlbXMkaXRlbTFfSURbaV0pXSwKICAgICAgICAgICAgICAgICAgICAgIGR1cGxpY2F0ZV9pdGVtc19kYXRhX3QyWyxjKGR1cGxpY2F0ZV9pdGVtcyRpdGVtMl9JRFtpXSldKSkkcmhvWzJdKQoKICBkdXBsaWNhdGVfaXRlbXMkdDJfdDFfcG9seWNvcltpXSA9IGFicyhwb2x5Y2hvcmljKGRhdGEuZnJhbWUoZHVwbGljYXRlX2l0ZW1zX2RhdGFfdDJbLGMoZHVwbGljYXRlX2l0ZW1zJGl0ZW0xX0lEW2ldKV0sCiAgICAgICAgICAgICAgICAgICAgICBkdXBsaWNhdGVfaXRlbXNfZGF0YV90MVssYyhkdXBsaWNhdGVfaXRlbXMkaXRlbTJfSURbaV0pXSkpJHJob1syXSkKfQoKZHVwbGljYXRlX2l0ZW1zCmBgYAoKYGBge3J9CnN1bW1hcnkoZHVwbGljYXRlX2l0ZW1zJHQxX3QxX3BvbHljb3IpCnN1bW1hcnkoZHVwbGljYXRlX2l0ZW1zJHQyX3QyX3BvbHljb3IpCnN1bW1hcnkoZHVwbGljYXRlX2l0ZW1zJHQxX3QyX3BvbHljb3IpCnN1bW1hcnkoZHVwbGljYXRlX2l0ZW1zJHQyX3QxX3BvbHljb3IpCmBgYAoKYGBge3IgZWNobz1GQUxTRX0Kcm0odG1wLCBjb21iLCBzaWdfZGF5c19lZmZlY3QsIHQxXzJfZGlmZmVyZW5jZSwgZHVwbGljYXRlX2l0ZW1zLCBkdXBsaWNhdGVfaXRlbXNfZGF0YV90MSwgZHVwbGljYXRlX2l0ZW1zX2RhdGFfdDIsIGV4dHJhY3RfaXRlbXMsIGdldF9kZWxheV9lZmZlY3QpCmBgYAoKIyMgQ29tcGFyaXNvbiB0byBwcmlvciBsaXRlcmF0dXJlCgpSZWFkIGluIGFuZCBwcm9jZXNzIGJvb3RzdHJhcHBlZCByZXN1bHRzLgoKYGBge3J9CnByb2Nlc3NfYm9vdF9kZiA9IGZ1bmN0aW9uKGRmKXsKICBkZiA9IGRmICU+JQogIGRyb3BfbmEoKSAlPiUKICBtdXRhdGUoZHYgPSBhcy5jaGFyYWN0ZXIoZHYpLAogICAgICAgICBpY2MgPSBhcy5udW1lcmljKGFzLmNoYXJhY3RlcihpY2MpKSwKICAgICAgICAgc3BlYXJtYW4gPSBhcy5udW1lcmljKGFzLmNoYXJhY3RlcihzcGVhcm1hbikpLAogICAgICAgICBwZWFyc29uID0gYXMubnVtZXJpYyhhcy5jaGFyYWN0ZXIocGVhcnNvbikpLAogICAgICAgICBldGFfc3EgPSBhcy5udW1lcmljKGFzLmNoYXJhY3RlcihldGFfc3EpKSwKICAgICAgICAgc2VtID0gYXMubnVtZXJpYyhhcy5jaGFyYWN0ZXIoc2VtKSksCiAgICAgICAgIHBhcnRpYWxfZXRhX3NxID0gYXMubnVtZXJpYyhhcy5jaGFyYWN0ZXIocGFydGlhbF9ldGFfc3EpKSwKICAgICAgICAgb21lZ2Ffc3EgPSBhcy5udW1lcmljKGFzLmNoYXJhY3RlcihvbWVnYV9zcSkpLAogICAgICAgICB2YXJfc3VicyA9IGFzLm51bWVyaWMoYXMuY2hhcmFjdGVyKHZhcl9zdWJzKSksCiAgICAgICAgIHZhcl9pbmQgPSBhcy5udW1lcmljKGFzLmNoYXJhY3Rlcih2YXJfaW5kKSksCiAgICAgICAgIHZhcl9yZXNpZCA9IGFzLm51bWVyaWMoYXMuY2hhcmFjdGVyKHZhcl9yZXNpZCkpLAogICAgICAgICBGX3RpbWUgPSBhcy5udW1lcmljKGFzLmNoYXJhY3RlcihGX3RpbWUpKSwKICAgICAgICAgcF90aW1lID0gYXMubnVtZXJpYyhhcy5jaGFyYWN0ZXIocF90aW1lKSksCiAgICAgICAgIGRmX3RpbWUgPSBhcy5udW1lcmljKGFzLmNoYXJhY3RlcihkZl90aW1lKSksCiAgICAgICAgIGRmX3Jlc2lkID0gYXMubnVtZXJpYyhhcy5jaGFyYWN0ZXIoZGZfcmVzaWQpKSkKICByZXR1cm4oZGYpfSAKCmJvb3RfZGYgPC0gcmVhZC5jc3YoZ3pmaWxlKHBhc3RlMChyZXRlc3RfZGF0YV9wYXRoLCdib290c3RyYXBfbWVyZ2VkLmNzdi5neicpKSkKCmJvb3RfZGYgPSBwcm9jZXNzX2Jvb3RfZGYoYm9vdF9kZikKCmJvb3RfZGYgPSBib290X2RmW2Jvb3RfZGYkZHYgJWluJSByZXRlc3RfcmVwb3J0X3ZhcnMsXQoKIyBDaGVjayBpZiB5b3UgaGF2ZSBhbGwgdmFyaWFibGVzIGJvb3RzdHJhcHBlZAojIHJldGVzdF9yZXBvcnRfdmFyc1t3aGljaChyZXRlc3RfcmVwb3J0X3ZhcnMgJWluJSBib290X2RmJGR2PT1GQUxTRSldCgojIEJvb3QgZGYgY29udGFpbnMgaGRkbSBwYXJhbWV0ZXJzIGZpdCBvbiB0aGUgZnVsbCBzYW1wbGUgaW4gdGhlIHQxIGRhdGEKIyByZWZpdHNfYm9vdHN0cmFwX21lcmdlZC5jc3YuZ3ogY29udGFpbnMgYm9vdHN0cmFwcGVkIHJlbGlhYmlsaXRpZXMgCgpyZWZpdF9ib290X2RmID0gcmVhZC5jc3YoZ3pmaWxlKHBhc3RlMChyZXRlc3RfZGF0YV9wYXRoLCdyZWZpdHNfYm9vdHN0cmFwX21lcmdlZC5jc3YuZ3onKSkpCgpyZWZpdF9ib290X2RmID0gcHJvY2Vzc19ib290X2RmKHJlZml0X2Jvb3RfZGYpCgpmdWxsZml0X2Jvb3RfZGYgPSBib290X2RmW2FzLmNoYXJhY3Rlcihib290X2RmJGR2KSAlaW4lIHVuaXF1ZShhcy5jaGFyYWN0ZXIocmVmaXRfYm9vdF9kZiRkdikpLF0KCmJvb3RfZGYgPSBib290X2RmWyFhcy5jaGFyYWN0ZXIoYm9vdF9kZiRkdikgJWluJSB1bmlxdWUoYXMuY2hhcmFjdGVyKHJlZml0X2Jvb3RfZGYkZHYpKSxdCgpib290X2RmID0gcmJpbmQoYm9vdF9kZiwgcmVmaXRfYm9vdF9kZikKCnJtKHJlZml0X2Jvb3RfZGYpCmBgYAoKU3VtbWFyaXplIGJvb3RzdHJhcHBlZCByZXN1bHRzIGFuZCBtZXJnZSB0byBsaXQgcmV2aWV3IGRhdGEKCmBgYHtyIG1lc3NhZ2U9RkFMU0V9CnZhcl9ib290X2RmID0gYm9vdF9kZiAlPiUKICBncm91cF9ieShkdikgJT4lCiAgc3VtbWFyaXNlKG1lYW5faWNjID0gbWVhbihpY2MpLAogICAgICAgICAgICBtZWFuX3BlYXJzb24gPSBtZWFuKHBlYXJzb24pKQoKcmVsX2NvbXAgPSBsaXRfcmV2aWV3ICU+JQogIGxlZnRfam9pbih2YXJfYm9vdF9kZiwgYnkgPSAnZHYnKQpgYGAKCkhlcmUncyB3aGF0IG91ciBkYXRhIGxvb2tzIGxpa2U6ICg1ODMgZGF0YSBwb2ludHMgZm9yIDE3MSBtZWFzdXJlcykKCmBgYHtyfQpyZWxfY29tcApgYGAKCiMjIyBMaXQgcmV2aWV3IHJlc3VsdHMKCkRpc3RyaWJ1dGlvbiBvZiByZWxpYWJpbGl0aWVzLCBzYW1wbGUgc2l6ZXMgYW5kIGRlbGF5cwoKYGBge3J9CnJlbF9jb21wICU+JSAKICBzZWxlY3QoZHYsIHRhc2ssIHJldGVzdF9yZWxpYWJpbGl0eSwgc2FtcGxlX3NpemUsIGRheXMpICU+JQogIGZpbHRlcihkYXlzIDwgMzYwMCkgJT4lCiAgZ2F0aGVyKGtleSwgdmFsdWUsIC1kdiwgLXRhc2spICU+JQogIGdncGxvdChhZXModmFsdWUsIGZpbGw9dGFzaykpKwogIGdlb21fZGVuc2l0eShhbHBoYT0wLjUsIHBvc2l0aW9uPSdpZGVudGl0eScpKwogIGZhY2V0X3dyYXAofmtleSwgc2NhbGVzPSdmcmVlJykrCiAgdGhlbWUobGVnZW5kLnRpdGxlID0gZWxlbWVudF9ibGFuaygpKQpgYGAKCmBgYHtyfQpzdW1tYXJ5KHJlbF9jb21wJHNhbXBsZV9zaXplW3JlbF9jb21wJHRhc2sgPT0gInN1cnZleSJdKQpzdW1tYXJ5KHJlbF9jb21wJHNhbXBsZV9zaXplW3JlbF9jb21wJHRhc2sgPT0gInRhc2siXSkKYGBgCgpUaGUgbGl0ZXJhdHVyZSBoYXMgc21hbGxlciBzaXplZCBzYW1wbGVzIGZvciB0YXNrIG1lYXN1cmVzIGNvbXBhcmVkIHRvIHN1cnZleSBtZWFzdXJlcyB0aGF0IHJlcG9ydCByZXRlc3QgcmVsaWFiaWxpdHkuCgpgYGB7cn0Kc3VtbWFyeShsbShzYW1wbGVfc2l6ZSB+IHRhc2sscmVsX2NvbXApKQpgYGAKCldoYXQgcHJlZGljdHMgcmV0ZXN0IHJlbGlhYmlsaXR5IGluIHRoZSBsaXRlcmF0dXJlPwpUYXNrLCBzYW1wbGUgc2l6ZSwgZGF5cwoKYGBge3J9Cm1vZDEgPSBsbWVyKHJldGVzdF9yZWxpYWJpbGl0eSB+IHRhc2sgKyAoMXxkdiksIHJlbF9jb21wKQptb2QyID0gbG1lcihyZXRlc3RfcmVsaWFiaWxpdHkgfiB0YXNrICsgc2FtcGxlX3NpemUgKyAoMXxkdiksIHJlbF9jb21wKQoKYW5vdmEobW9kMSwgbW9kMikKYGBgCgpgYGB7cn0KbW9kMyA9IGxtZXIocmV0ZXN0X3JlbGlhYmlsaXR5IH4gdGFzayArIHNhbXBsZV9zaXplICsgZGF5cysgKDF8ZHYpLCByZWxfY29tcCkKCmFub3ZhKG1vZDIsIG1vZDMpCmBgYAoKYGBge3J9CnN1bW1hcnkobW9kMikKYGBgCgpUYXNrcyBoYXZlIHNpZ25pZmljYW50bHkgbG93ZXIgcmVsaWFiaWxpdHkgYW5kIHJlbGlhYmlsaXR5IGRlY3JlYXNlcyB3aXRoIGluY3JlYXNpbmcgc2FtcGxlIHNpemUuCgpgYGB7cn0KcmVsX2NvbXAgJT4lIAogIGdncGxvdChhZXMoc2FtcGxlX3NpemUsIHJldGVzdF9yZWxpYWJpbGl0eSwgY29sb3I9dGFzaykpKwogIGdlb21fc21vb3RoKG1ldGhvZD0nbG0nKSsKICBnZW9tX3BvaW50KGFscGhhID0gMC4yKSsKICB0aGVtZShsZWdlbmQudGl0bGUgPSBlbGVtZW50X2JsYW5rKCkpKwogIHhsYWIoIlNhbXBsZSBTaXplIikrCiAgeWxhYigiUmV0ZXN0IHJlbGlhYmlsaXR5IikKYGBgCgpJIHVzZWQgdG8gY29tcGFyZSBlZmZlY3Qgc2l6ZXMgb2YgZWZmZWN0IG9mIHRhc2sgb24gcmVsaWFiaWxpdHkgZXN0aW1hdGVzIGluIGxpdGVyYXR1cmUgdnMgb3VyIHJlc3VsdHMgKGp1c3QgbG9va2luZyBhdCB0aGUgdmFyaWFibGVzIHlvdSBmaW5kIGluIHRoZSBsaXRlcmF0dXJlKSBidXQgcmVtb3ZlZCB0aGlzIGFuYWx5c2lzIGJlY2F1c2UgICAKMS4gdGhlIGVzdGltYXRlcyBpbiB0aGUgbGl0ZXJhdHVyZSBhcmUgbm90IGFsbCB0aGUgc2FtZSBzdGF0aXN0aWMKMi4gZnJvbSBvdXIgZGF0YSBJIHdhcyBvbmx5IHVzaW5nIElDQydzCjMuIEkgd2FzIHNhbXBsaW5nIGZyb20gb3VyIGJvb3RzdHJhcHBlZCByZXN1bHRzIGFzIG1hbnkgZGF0YXBvaW50cyBmb3IgZWFjaCBtZWFzdXJlIGFzIHRoZXJlIGFyZSBpbiB0aGUgbGl0ZXJhdHVyZSBidXQgdGhhdCBkaWRuJ3Qgc2VlbSBsaWtlIHRoZSBiZXN0IHdheSB0byBtYWtlIHVzZSBvZiBvdXIgZGF0YSB0byBtYWtlIGl0IGNvbXBhcmFibGUgdG8gdGhlIGxpdGVyYXR1cmUuCgpEZXNwaXRlIHRoZXNlIHByb2JsZW1zIHRoZSBtYWluIHJlc3VsdCB3YXMgdGhhdCB0aGUgZWZmZWN0IHNpemUgd2FzIG11Y2ggbGFyZ2VyIGluIG91ciBkYXRhc2V0IGNvbXBhcmVkIHRvIHRoZSBsaXRlcmF0dXJlIGJ1dCBnaXZlbiB0aGUgcHJvYmxlbXMgSSB0aGluayB0aGUgc2FtcGxpbmcgYW5hbHlzZXMgYmVsb3cgYXJlIG1vcmUgaW5mb3JtYXRpdmUgdGhhbiB0aGlzLgoKV2UgYWxzbyBjaGVja2VkIHdoZXRoZXIgb3VyIHJlc3VsdHMgZGl2ZXJnZSBtb3N0IGZyb20gc3R1ZGllcyB3aXRoIHNtYWxsZXIgc2FtcGxlIHNpemVzLiBTcXVhcmUgZGlmZmVyZW5jZSBiZXR3ZWVuIG91ciBtZWFuIGVzdGltYXRlIGFuZCB0aGUgcmVsaWFiaWxpdHkgZnJvbSB0aGUgbGl0ZXJhdHVyZSBkZWNyZWFzZXMgZXhwb25lbnRpYWxseSB3aXRoIHNhbXBsZSBzaXplLiBUaGUgc21hbGxlciB0aGUgc2FtcGxlIHNpemUgaW4gdGhlIGxpdGVyYXR1cmUgdGhlIG1vcmUgdGhlIHJlbGlhYmlsaXR5IGVzdGltYXRlIGRpZmZlcnMgZnJvbSBvdXIgcmVzdWx0cy4gQnV0IHRoaXMgd2FzIGEgd2VhayByZXN1bHQgYmVjYXVzZSBtb3N0IG9mIHRoZSBzdHVkaWVzIGluIHRoZSBsaXRlcmF0dXJlIGhhdmUgc21hbGxlciBzYW1wbGUgc2l6ZXMgYW5kIHlvdSBzZWUgYm90aCBzbWFsbCBhbmQgbGFyZ2UgZGV2aWF0aW9ucyBmb3IgdGhlc2Ugc3R1ZGllcyAodGhlc2Ugd2VyZSBub3Qgc2lnbmlmaWNhbnQgZWl0aGVyKS4KCiMjIyBEaXJlY3QgcmVsYXRpb24gdG8gb3VyIHJlc3VsdHMKCkNvcnJlbGF0aW9uIGJldHdlZW4gb3VyIG1lYW4gZXN0aW1hdGVzIGZyb20gYm9vdHN0cmFwcGVkIHNhbXBsZXMgYW5kIHRoZSBsaXRlcmF0dXJlIHJldmlldyBmb3IgdGFzayB2YXJpYWJsZXMKCmBgYHtyfQpuX2RmID0gcmVsX2NvbXAgJT4lIAogIGdyb3VwX2J5KGR2KSAlPiUKICB0YWxseSgpCgpsaXRfZW1wX2NvciA9IGZ1bmN0aW9uKCl7CiAgCiAgYm9vdF9jb21wID0gZGF0YS5mcmFtZSgpCiAgCiAgZm9yKGkgaW4gMTpsZW5ndGgodW5pcXVlKHJlbF9jb21wJGR2KSkgKXsKICAgIGN1cl9kdiA9IHVuaXF1ZShyZWxfY29tcCRkdilbaV0KICAgIG4gPSBuX2RmJG5bbl9kZiRkdiA9PSBjdXJfZHZdCiAgICBzYW1wbGVfZGYgPSBib290X2RmICU+JSBmaWx0ZXIoZHYgPT0gY3VyX2R2KQogICAgdG1wID0gc2FtcGxlX24oc2FtcGxlX2RmLCBuKQogICAgYm9vdF9jb21wID0gcmJpbmQoYm9vdF9jb21wLCB0bXApCiAgfSAgCiAgCiAgcm0oY3VyX2R2LCBuLCBzYW1wbGVfZGYsIHRtcCkKICAKICAjY2hlY2sgaWYgY2JpbmQgaXMgb2sKICAjIHN1bShib290X2NvbXAkZHYgPT0gcmVsX2NvbXAkZHYpCiAgI2NiaW5kaW5nIHBlYXJzb24gYmVjYXVzZSB0aGF0IGlzIHRoZSBtb3N0IGNvbW1vbiBtZXRyaWMgaW4gdGhlIGxpdAogIHJlbF9jb21wID0gY2JpbmQocmVsX2NvbXAsIGJvb3RfY29tcCRwZWFyc29uKQogICNyZW5hbWUgbmV3IGNvbHVtbgogIG5hbWVzKHJlbF9jb21wKVt3aGljaChuYW1lcyhyZWxfY29tcCkgPT0gImJvb3RfY29tcCRwZWFyc29uIildID0gInBlYXJzb24iCiAgCiAgb3V0ID0gZGF0YS5mcmFtZSh0YXNrID0gTkEsIHN1cnZleSA9IE5BKQogIAogIG91dCR0YXNrID0gd2l0aChyZWxfY29tcCAlPiUgZmlsdGVyKHRhc2sgPT0gInRhc2siKSwgY29yKHBlYXJzb24sIHJldGVzdF9yZWxpYWJpbGl0eSkpCiAgCiAgb3V0JHN1cnZleSA9IHdpdGgocmVsX2NvbXAgJT4lIGZpbHRlcih0YXNrID09ICJzdXJ2ZXkiKSwgY29yKHBlYXJzb24sIHJldGVzdF9yZWxpYWJpbGl0eSkpCiAgCiAgcmVsX2NvbXAgPSByZWxfY29tcFssLTE2XQogIAogIHJldHVybihvdXQpCn0KCmxpdF9lbXBfY29yX291dCA9IHBseXI6OnJkcGx5KDEwMCwgbGl0X2VtcF9jb3IpCgp3cml0ZS5jc3YobGl0X2VtcF9jb3Jfb3V0LCcvVXNlcnMvemV5bmVwZW5rYXZpL0Ryb3Bib3gvUG9sZHJhY2tMYWIvU1JPX1JldGVzdF9BbmFseXNlcy9vdXRwdXQvdGFibGVzL2xpdF9lbXBfY29yX291dC5jc3YnKQoKc3VtbWFyeShsaXRfZW1wX2Nvcl9vdXQpCgpgYGAKCiMjIyBOb2lzZSBjZWlsaW5nCgpNb2RlbCBjb21wYXJpc29ucyBidWlsZGluZyBtb2RlbCB0byBwcmVkaWN0IHRoZSByZWxpYWJpbGl0aWVzIGluIHRoZSBsaXRlcmF0dXJlIGZyb20gYSBzYW1wbGUgZnJvbSBvdXIgcmVzdWx0cyB2ZXJzdXMgYSBzYW1wbGUgZm9ybSB0aGUgbGl0ZXJhdHVyZS4KCnNhbXBsZSBvbmUgcm93IHBlciBtZWFzdXJlIG91dCBvZiBsaXQgcmV2aWV3CnJeMiBvZiByZXRlc3RfcmVsaWFiaWxpdHkgfiBzYW1wbGVkX3JlbGlhYmlsaXR5IHZzLiAKcl4yIG9mIHJldGVzdF9yZWxpYWJpbGl0eSB+IG1lYW5faWNjCgpjb2VmIG9mIHJldGVzdF9yZWxpYWJpbGl0eSB+IHNhbXBsZWRfcmVsaWFiaWxpdHkgdnMuIApjb2VmIG9mIHJldGVzdF9yZWxpYWJpbGl0eSB+IG1lYW5faWNjCgpgYGB7cn0KY29tcF9saXRfcHJlZCA8LSBmdW5jdGlvbihkZil7CiAgCiAgc2FtcGxlX2Zyb21fZHYgPC0gZnVuY3Rpb24oZGYpewogICAgaWYobnJvdyhkZik+MSl7CiAgICAgIHJvd19udW0gPSBzYW1wbGUoMTpucm93KGRmKSwxKQogICAgICBzYW1wbGVfcm93ID0gZGZbcm93X251bSxdCiAgICAgIGRmID0gZGZbLXJvd19udW0sXQogICAgICBkZiRsaXRfcHJlZGljdG9yID0gc2FtcGxlX3JvdyRyZXRlc3RfcmVsaWFiaWxpdHkKICAgIH0KICAgIHJldHVybihkZikKICB9CiAgCiAgc2FtcGxlZF9kZiA9IGRmICU+JQogICAgZ3JvdXBfYnkoZHYpICU+JQogICAgZG8oc2FtcGxlX2Zyb21fZHYoLikpIAogIAogIG1vZF9saXQgPSBsbShyZXRlc3RfcmVsaWFiaWxpdHkgfiBsaXRfcHJlZGljdG9yK3NjYWxlKHNhbXBsZV9zaXplKSt0YXNrLCBkYXRhPXNhbXBsZWRfZGYpCiAgbW9kX2Jvb3QgPSBsbShyZXRlc3RfcmVsaWFiaWxpdHkgfiBtZWFuX3BlYXJzb24rc2NhbGUoc2FtcGxlX3NpemUpK3Rhc2ssIGRhdGE9c2FtcGxlZF9kZikKICAKICBvdXQgPSBkYXRhLmZyYW1lKHIyX2xpdCA9IHN1bW1hcnkobW9kX2xpdCkkci5zcXVhcmVkLAogICAgICAgICAgICAgICAgICAgcjJfYm9vdCA9IHN1bW1hcnkobW9kX2Jvb3QpJHIuc3F1YXJlZCwKICAgICAgICAgICAgICAgICAgIG1fbGl0ID0gY29lZihzdW1tYXJ5KG1vZF9saXQpKVsibGl0X3ByZWRpY3RvciIsIkVzdGltYXRlIl0sCiAgICAgICAgICAgICAgICAgICBtX2Jvb3QgPSBjb2VmKHN1bW1hcnkobW9kX2Jvb3QpKVsibWVhbl9wZWFyc29uIiwiRXN0aW1hdGUiXSkKICAKICByZXR1cm4ob3V0KQp9Cgpjb21wX2xpdF9wcmVkX291dCA9IHBseXI6OnJkcGx5KDEwMDAsIGNvbXBfbGl0X3ByZWQocmVsX2NvbXApKQpgYGAKCgpgYGB7cn0KdG1wID0gY29tcF9saXRfcHJlZF9vdXQgJT4lCiAgc2VsZWN0KC0ubikgJT4lCiAgZ2F0aGVyKGtleSwgdmFsdWUpICU+JQogIHNlcGFyYXRlKGtleSwgYygic3RhdCIsICJzYW1wbGUiKSwgc2VwID0gIl8iKQogCnRtcCRzdGF0ID0gYXMuZmFjdG9yKHRtcCRzdGF0KQpsZXZlbHModG1wJHN0YXQpIDwtIGMoImNvZWZmaWNpZW50IiwgZXhwcmVzc2lvbihyXiIyIikpCgoKdG1wICU+JSAgCiAgZ2dwbG90KGFlcyh2YWx1ZSwgZmlsbD1zYW1wbGUpKSsKICBnZW9tX2RlbnNpdHkoYWxwaGEgPSAwLjUsIHBvc2l0aW9uPSdpZGVudGl0eScsIGNvbG9yPU5BKSsKICBmYWNldF9ncmlkKC5+c3RhdCwgc2NhbGVzPSdmcmVlJywgbGFiZWxsZXIgPSBsYWJlbF9wYXJzZWQpKwogIHNjYWxlX2ZpbGxfZGlzY3JldGUoYnJlYWtzPWMoImJvb3QiLCJsaXQiKSwKICAgICAgICAgICAgICAgICAgICAgICBsYWJlbHM9YygiRW1waXJpY2FsIiwgIkxpdGVyYXR1cmUiKSwKICAgICAgICAgICAgICAgICAgICAgIG5hbWU9IlByZWRpY3Rpb24iKSsKICB4bGFiKCcnKSsKICB5bGFiKCcnKQoKZ2dzYXZlKCdMaXRfTm9pc2VfQ2VpbGluZy5qcGcnLCBkZXZpY2UgPSAianBlZyIsIHBhdGggPSAiL1VzZXJzL3pleW5lcGVua2F2aS9Ecm9wYm94L1BvbGRyYWNrTGFiL1NST19SZXRlc3RfQW5hbHlzZXMvb3V0cHV0L2ZpZ3VyZXMvIiwgd2lkdGggPSAxMCwgaGVpZ2h0ID0gNCwgdW5pdHMgPSAiaW4iLCBsaW1pdHNpemUgPSBGQUxTRSwgZHBpID0gMTAwKQpgYGAKCgpgYGB7cn0Kd2l0aChjb21wX2xpdF9wcmVkX291dCwgdC50ZXN0KHIyX2xpdCwgcjJfYm9vdCwgcGFpcmVkPVQpKQpgYGAKCmBgYHtyfQp3aXRoKGNvbXBfbGl0X3ByZWRfb3V0LCB0LnRlc3QobV9saXQsIG1fYm9vdCkpCmBgYAoKYGBge3IgZWNobz1GQUxTRX0Kcm0odG1wLCByZWxfY29tcCwgbW9kMSwgbW9kMiwgbW9kMywgbl9kZiwgaSwgbGl0X2VtcF9jb3IsIGxpdF9lbXBfY29yX291dCwgY29tcF9saXRfcHJlZCwgY29tcF9saXRfcHJlZF9vdXQpCmBgYAoKIyMgUmVsYXRpb25zaGlwIGJldHdlZW4gcmVsaWFiaWxpdHkgbWV0cmljcyAocG9pbnQgZXN0aW1hdGVzKQoKQmFzZWQgb24gW1dlaXIgKDIwMDUpXShodHRwczovL3BkZnMuc2VtYW50aWNzY2hvbGFyLm9yZy9kOTlhLzc5MGNjZTQzZjdmMjBkNzQyZjlkMzc5Yjc5ZGU0Zjc2Nzc0MC5wZGYpIElDQygzLGspIGRvZXMgbm90IHRha2UgaW4gdG8gYWNjb3VudCB3aXRoaW4gc3ViamVjdCBkaWZmZXJlbmNlcyBiZXR3ZWVuIHR3byB0aW1lIHBvaW50cyAoaS5lLiB0aGUgZml4ZWQgZWZmZWN0IG9mIHRpbWUvc3lzdGVtYXRpYyBlcnJvcikuIFRodXMsIGl0IGlzIHdlbGwgYXBwcm94aW1hdGVkIGJ5IFBlYXJzb24ncyByIGFuZCBzdWJqZWN0IHRvIHNpbWlsYXIgY3JpdGljaXNtcy4gW1dlaXIgKDIwMDUpXShodHRwczovL3BkZnMuc2VtYW50aWNzY2hvbGFyLm9yZy9kOTlhLzc5MGNjZTQzZjdmMjBkNzQyZjlkMzc5Yjc5ZGU0Zjc2Nzc0MC5wZGYpIHN1Z2dlc3RzIHJlcG9ydGluZyBhdCBsZWFzdCB0aGlzIHN5c3RlbWF0aWMgZXJyb3IgZWZmZWN0IHNpemUgaWYgb25lIGNob29zZXMgdG8gcmVwb3J0IHdpdGggSUNDKDMsaykuIEJhc2VkIG9uIGhpcyBjb25jbHVzaW9ucyBoZXJlIEkgcmVwb3J0OiAgCi0gSUNDKDMsayk6IEFzIERhdmUgY2xhcmlmaWVkIHRoaXMgcmFuZ2VzIGZyb20gMSB0byAtMS8obnVtYmVyIG9mIHJlcGVhdGVkIG1lYXN1cmVzIC0xKSBzbyBpbiBvdXIgY2FzZSB0aGlzIHJhbmdlIHdvdWxkIGJlIFstMSwgMV07IGxhcmdlciB2YWx1ZXMgd291bGQgbWVhbiB0aGF0IHRoZSB0d28gc2NvcmVzIG9mIGEgc3ViamVjdCBmb3IgYSBnaXZlbiBtZWFzdXJlIGFyZSBtb3JlIHNpbWlsYXIgdG8gZWFjaCBvdGhlciB0aGFuIHRoZXkgYXJlIHRvIHNjb3JlcyBvZiBvdGhlciBwZW9wbGUgIAotICJJQ0MgaXMgcmVmbGVjdGl2ZSBvZiB0aGUgYWJpbGl0eSBvZiBhIHRlc3QgdG8gZGlmZmVyZW50aWF0ZSBiZXR3ZWVuIGRpZmZlcmVudCBpbmRpdmlkdWFscyIKLSBwYXJ0aWFsICRcZXRhXjIkIGZvciB0aW1lICgkU1Nfe3RpbWV9L1NTX3t3aXRoaW59JCk6IGVmZmVjdCBzaXplIG9mIHRpbWUgICAKLSBTRU0gKCRcc3FydChNU197ZXJyb3J9KSQpOiBzdGFuZGFyZCBlcnJvciBvZiBtZWFzdXJlbWVudDsgdGhlIHNtYWxsZXIgdGhlIGJldHRlci4gSXQgCiJxdWFudGlmaWVzIHRoZSBwcmVjaXNpb24gb2YgaW5kaXZpZHVhbCBzY29yZXMgb24gYSB0ZXN0IiBhbmQgaXMgbm90IGRlcGVuZGVudCBvbiB0aGUgc2FtcGxlIGluIHRoZSB3YXkgSUNDIGlzIHNpbmNlIGl0IGRvZXNuJ3QgZGVwZW5kIG9uIGJldHdlZW4gc3ViamVjdCByZWxpYWJpbGl0eSAoYXQgbGVhc3QgaW4gdGhpcyBmb3JtdWxhdGlvbikgYnV0IGlzIHVuaXQtZGVwZW5kZW50LgoKV2UgY2FsY3VsYXRlZCBgciBhcy5udW1lcmljKHRhYmxlKHJlbF9kZiR0YXNrKVsxXSlgIG1lYXN1cmVzIGZvciBzdXJ2ZXlzIGFuZCBgciBhcy5udW1lcmljKHRhYmxlKHJlbF9kZiR0YXNrKVsyXSlgIG1lYXN1cmVzIGZvciBjb2duaXRpdmUgdGFza3MuICAKClRob3VnaCB3ZSBhcmUgcHJpbWFyaWx5IHJlcG9ydGluZyBJQ0MncyBhcyBvdXIgbWV0cmljIG9mIHJlbGlhYmlsaXR5IHRoZSByZXN1bHRzIGRvbid0IGNoYW5nZSBkZXBlbmRpbmcgb24gdGhlIG1ldHJpYyBjaG9zZW4uIEhlcmUgd2UgcGxvdCBwb2ludCBlc3RpbWF0ZXMgb2YgdGhyZWUgZGlmZmVyZW50IHJlbGlhYmlsaXR5IG1ldHJpY3MgYWdhaW5zdCBlYWNoIG90aGVyIChJQ0MsIFBlYXJzb24sIFNwZWFybWFuKS4gVGhlIGJvb3RzdHJhcHBlZCB2ZXJzaW9uIGlzIGVzc2VudGlhbGx5IHRoZSBzYW1lIGJ1dCB0aGUgcGxvdHMgYXJlIGJ1c2llciBkdWUgdG8gbW9yZSBkYXRhcG9pbnRzLgoKYGBge3J9CnRhYmxlKHJlbF9kZiR0YXNrKQpgYGAKCmBgYHtyIHdhcm5pbmc9RkFMU0UsIG1lc3NhZ2U9RkFMU0V9CnAxID0gcmVsX2RmICU+JQogIGdncGxvdChhZXMoc3BlYXJtYW4sIGljYywgY29sPXRhc2spKSsKICBnZW9tX3BvaW50KCkrCiAgdGhlbWVfYncoKSsKICB0aGVtZShsZWdlbmQudGl0bGUgPSBlbGVtZW50X2JsYW5rKCksCiAgICAgICAgbGVnZW5kLnBvc2l0aW9uID0gIm5vbmUiKSsKICBnZW9tX2FibGluZShpbnRlcmNlcHQgPSAwLCBzbG9wZT0xKQoKcDIgPSByZWxfZGYgJT4lCiAgZ2dwbG90KGFlcyhwZWFyc29uLCBpY2MsIGNvbD10YXNrKSkrCiAgZ2VvbV9wb2ludCgpKwogIHRoZW1lX2J3KCkrCiAgdGhlbWUobGVnZW5kLnRpdGxlID0gZWxlbWVudF9ibGFuaygpLAogICAgICAgIGxlZ2VuZC5wb3NpdGlvbiA9ICJub25lIikrCiAgZ2VvbV9hYmxpbmUoaW50ZXJjZXB0ID0gMCwgc2xvcGU9MSkKCnAzID0gcmVsX2RmICU+JQogIGdncGxvdChhZXMocGVhcnNvbiwgc3BlYXJtYW4sIGNvbD10YXNrKSkrCiAgZ2VvbV9wb2ludCgpKwogIHRoZW1lX2J3KCkrCiAgdGhlbWUobGVnZW5kLnRpdGxlID0gZWxlbWVudF9ibGFuaygpLAogICAgICAgIGxlZ2VuZC5wb3NpdGlvbiA9ICJub25lIikrCiAgZ2VvbV9hYmxpbmUoaW50ZXJjZXB0ID0gMCwgc2xvcGU9MSkKCmdyaWQuYXJyYW5nZShwMSwgcDIsIHAzLCBucm93PTEpCgpnZ3NhdmUoJ01ldHJpY19TY2F0dGVycGxvdHMuanBnJywgcGxvdCA9IGdyaWQuYXJyYW5nZShwMSwgcDIsIHAzLCBucm93PTEpLCBkZXZpY2UgPSAianBlZyIsIHBhdGggPSAiL1VzZXJzL3pleW5lcGVua2F2aS9Ecm9wYm94L1BvbGRyYWNrTGFiL1NST19SZXRlc3RfQW5hbHlzZXMvb3V0cHV0L2ZpZ3VyZXMvIiwgd2lkdGggPSAxMiwgaGVpZ2h0ID0gNCwgdW5pdHMgPSAiaW4iLCBsaW1pdHNpemUgPSBGQUxTRSwgZHBpID0gMTAwKQpgYGAKCkFzIHRoZSBzY2F0dGVyIHBsb3RzIGRlcGljdCB0aGUgY29ycmVsYXRpb25zIGJldHdlZW4gZGlmZmVyZW50IHR5cGVzIHJlbGlhYmlsaXR5IG1ldHJpY3Mgd2VyZSB2ZXJ5IGhpZ2guCgpgYGB7cn0KY29yKHJlbF9kZlssYygnc3BlYXJtYW4nLCAnaWNjJywgJ3BlYXJzb24nKV0pCmBgYAoKYGBge3IgZWNobz1GQUxTRX0Kcm0ocDEscDIscDMpCmBgYAoKTm90ZTogU29tZSB2YXJpYWJsZXMgaGF2ZSA8MCBJQ0Mncy4gVGhpcyB3b3VsZCBiZSB0aGUgY2FzZSBpZiB0aGUgJE1TX3tlcnJvcn0kPiRNU197YmV0d2Vlbn0kLiBEYXRhIGZvciB0aGVzZSB2YXJpYWJsZXMgaGF2ZSBubyByZWxhdGlvbnNoaXAgYmV0d2VlbiB0aGUgdHdvIHRpbWUgcG9pbnRzLgoKIyMgU3VtbWFyeSBvZiBhbGwgbWVhc3VyZSByZWxpYWJpbGl0aWVzCgpTdW1tYXJpemVkIGJvb3RzdHJhcHBlZCByZWxpYWJpbGl0aWVzCgpgYGB7ciBtZXNzYWdlPUZBTFNFLCB3YXJuaW5nPUZBTFNFfQpib290X2RmICU+JQogIGdyb3VwX2J5KGR2KSAlPiUKICBzdW1tYXJpc2UoaWNjX21lZGlhbiA9IHF1YW50aWxlKGljYywgcHJvYnMgPSAwLjUpLAogICAgICAgICAgICBpY2NfMi41ID0gcXVhbnRpbGUoaWNjLCBwcm9icyA9IDAuMDI1KSwKICAgICAgICAgICAgaWNjXzk3LjUgPSBxdWFudGlsZShpY2MsIHByb2JzID0gMC45NzUpLAogICAgICAgICAgICBzcGVhcm1hbl9tZWRpYW4gPSBxdWFudGlsZShzcGVhcm1hbiwgcHJvYnMgPSAwLjUpLAogICAgICAgICAgICBzcGVhcm1hbl8yLjUgPSBxdWFudGlsZShzcGVhcm1hbiwgcHJvYnMgPSAwLjAyNSksCiAgICAgICAgICAgIHNwZWFybWFuXzk3LjUgPSBxdWFudGlsZShzcGVhcm1hbiwgcHJvYnMgPSAwLjk3NSkpICU+JQogIGRhdGF0YWJsZSgpICU+JQogIGZvcm1hdFJvdW5kKGNvbHVtbnM9YygnaWNjX21lZGlhbicsICdpY2NfMi41JywgJ2ljY185Ny41JywgJ3NwZWFybWFuX21lZGlhbicsICdzcGVhcm1hbl8yLjUnLCAnc3BlYXJtYW5fOTcuNScpLCBkaWdpdHM9MykKYGBgCgpgYGB7ciBlY2hvPUZBTFNFfQptZWFzdXJlX2xhYmVscyA8LSByZWFkLmNzdignL1VzZXJzL3pleW5lcGVua2F2aS9Ecm9wYm94L1BvbGRyYWNrTGFiL1NST19SZXRlc3RfQW5hbHlzZXMvaW5wdXQvbWVhc3VyZV9sYWJlbHMuY3N2JykKbWVhc3VyZV9sYWJlbHMgPSBtZWFzdXJlX2xhYmVscyAlPiUgc2VsZWN0KC1tZWFzdXJlX2Rlc2NyaXB0aW9uKQojIENoZWNrIGlmIHRoZXJlIGFyZSBhbnkgbWlzc2luZyB2YXJpYWJsZXMKIyByZXRlc3RfcmVwb3J0X3ZhcnNbKHJldGVzdF9yZXBvcnRfdmFycyAlaW4lIG1lYXN1cmVfbGFiZWxzJGR2ID09IEZBTFNFKV0KIyBtZWFzdXJlX2xhYmVscyRkdlsobWVhc3VyZV9sYWJlbHMkZHYgJWluJSByZXRlc3RfcmVwb3J0X3ZhcnMgPT0gRkFMU0UpXQpgYGAKCmBgYHtyIHdhcm5pbmc9RkFMU0UsIG1lc3NhZ2U9RkFMU0V9CiMgRGYgd3JhbmdsaW5nIGZvciBwbG90dGluZwp0bXAgPSBtZWFzdXJlX2xhYmVscyAlPiUKICBtdXRhdGUoZHYgPSBhcy5jaGFyYWN0ZXIoZHYpKSAlPiUKICBsZWZ0X2pvaW4oYm9vdF9kZlssYygiZHYiLCAiaWNjIiwgInNwZWFybWFuIildLCBieSA9ICdkdicpIAoKdG1wID0gdG1wICU+JQogIHNlcGFyYXRlKGR2LCBjKCJ0YXNrX2dyb3VwIiwgInZhciIpLCBzZXA9IlxcLiIscmVtb3ZlPUZBTFNFLGV4dHJhPSJtZXJnZSIpICU+JQogIG11dGF0ZSh0YXNrX2dyb3VwID0gZmFjdG9yKHRhc2tfZ3JvdXAsIGxldmVscyA9IHRhc2tfZ3JvdXBbb3JkZXIodGFzayldKSkgJT4lCiAgc2VwYXJhdGUodmFyLCBjKCJ2YXIiKSwgc2VwPSJcXC4iLHJlbW92ZT1UUlVFLGV4dHJhPSJkcm9wIikgJT4lCiAgbXV0YXRlKHRhc2tfZ3JvdXAgPSBnc3ViKCJfIiwgIiAiLCB0YXNrX2dyb3VwKSwKICAgICAgICAgdmFyID0gZ3N1YigiXyIsICIgIiwgdmFyKSkgJT4lCiAgYXJyYW5nZSh0YXNrX2dyb3VwLCB2YXIpCgp0bXAgPSB0bXAgJT4lCiAgbGVmdF9qb2luKHJlbF9kZlssYygiZHYiLCAiaWNjIildLCBieSA9ICJkdiIpICU+JQogIHJlbmFtZShpY2MgPSBpY2MueCwgcG9pbnRfZXN0ID0gaWNjLnkpCgojTWFudWFsIGNvcnJlY3Rpb24KdG1wID0gdG1wICU+JQogIG11dGF0ZSh0YXNrID0gaWZlbHNlKHRhc2tfZ3JvdXAgPT0gJ2hvbHQgbGF1cnkgc3VydmV5JywgInRhc2siLCBhcy5jaGFyYWN0ZXIodGFzaykpKSAlPiUKICBtdXRhdGUodGFza19ncm91cCA9IGlmZWxzZSh0YXNrX2dyb3VwID09ICJwc3ljaG9sb2dpY2FsIHJlZnJhY3RvcnkgcGVyaW9kIHR3byBjaG9pY2VzIiwgInBzeWNob2xvZ2ljYWwgcmVmcmFjdG9yeSBwZXJpb2QiLCBpZmVsc2UodGFza19ncm91cCA9PSAiYW5nbGluZyByaXNrIHRhc2sgYWx3YXlzIHN1bm55IiwgImFuZ2xpbmcgcmlzayB0YXNrIix0YXNrX2dyb3VwKSkpICU+JQogIG11dGF0ZSh0YXNrX2dyb3VwID0gZ3N1Yigic3VydmV5IiwgIiIsIHRhc2tfZ3JvdXApKQpgYGAKClZhcmlhYmxlIGxldmVsIHN1bW1hcnkgb2YgYm9vdHN0cmFwcGVkIHJlbGlhYmlsaXRpZXMuCgpgYGB7ciB3YXJuaW5nPUZBTFNFLCBtZXNzYWdlPUZBTFNFfSAKcDQgPC0gdG1wICU+JQogIGZpbHRlcih0YXNrID09ICd0YXNrJywKICAgICAgICAgcmF3X2ZpdCA9PSAncmF3JykgJT4lCmdncGxvdChhZXMoeSA9IHZhciwgeCA9IGljYykpICsgCiAgZ2VvbV9wb2ludChjb2xvciA9ICcjMDBCRkM0JykrCiAgZ2VvbV9wb2ludChhZXMoeSA9IHZhciwgeCA9IHBvaW50X2VzdCksIGNvbG9yID0gImJsYWNrIikrCiAgZmFjZXRfZ3JpZCh0YXNrX2dyb3Vwfi4sIHN3aXRjaCA9ICJ5Iiwgc2NhbGVzID0gImZyZWVfeSIsIHNwYWNlID0gImZyZWVfeSIsIGxhYmVsbGVyID0gbGFiZWxfd3JhcF9nZW4od2lkdGg9MjApKSArCiAgdGhlbWUocGFuZWwuc3BhY2luZyA9IHVuaXQoMC43NSwgImxpbmVzIiksIAogICAgICAgIHN0cmlwLnBsYWNlbWVudCA9ICJvdXRzaWRlIiwKICAgICAgICBzdHJpcC50ZXh0LnkgPSBlbGVtZW50X3RleHQoYW5nbGU9MTgwLCBzaXplPTM2KSwKICAgICAgICBheGlzLnRleHQueSA9IGVsZW1lbnRfdGV4dChzaXplPTIwKSwKICAgICAgICBwYW5lbC5iYWNrZ3JvdW5kID0gZWxlbWVudF9yZWN0KGZpbGwgPSBOQSksCiAgICAgICAgcGFuZWwuZ3JpZC5tYWpvciA9IGVsZW1lbnRfbGluZShjb2xvdXIgPSAiZ3JleTgwIikpICsgCiAgeGxhYigiIikrCiAgeWxhYigiIikrCiAgc2NhbGVfeF9jb250aW51b3VzKGxpbWl0cyA9IGMoLTAuMjUsMSksIGJyZWFrcz1jKC0wLjI1LCAwLCAwLjI1LCAwLjUsIDAuNzUsIDEpKSsKICBzY2FsZV95X2Rpc2NyZXRlKGxhYmVscyA9IGZ1bmN0aW9uKHgpIHN0cl93cmFwKHgsIHdpZHRoID0gMTApKSsKICBnZW9tX3ZsaW5lKHhpbnRlcmNlcHQgPSAwLCBjb2xvciA9ICJyZWQiLCBzaXplID0gMSkKCnA1IDwtIHRtcCAlPiUKICBmaWx0ZXIodGFzayA9PSAnc3VydmV5JykgJT4lCmdncGxvdChhZXMoeSA9IHZhciwgeCA9IGljYykpICsgCiAgZ2VvbV9wb2ludChjb2xvciA9ICcjRjg3NjZEJykrCiAgZ2VvbV9wb2ludChhZXMoeSA9IHZhciwgeCA9IHBvaW50X2VzdCksIGNvbG9yID0gImJsYWNrIikrCiAgZmFjZXRfZ3JpZCh0YXNrX2dyb3Vwfi4sIHN3aXRjaCA9ICJ5Iiwgc2NhbGVzID0gImZyZWVfeSIsIHNwYWNlID0gImZyZWVfeSIsIGxhYmVsbGVyID0gbGFiZWxfd3JhcF9nZW4od2lkdGg9MjApKSArCiAgdGhlbWUocGFuZWwuc3BhY2luZyA9IHVuaXQoMC43NSwgImxpbmVzIiksIAogICAgICAgIHN0cmlwLnBsYWNlbWVudCA9ICJvdXRzaWRlIiwKICAgICAgICBzdHJpcC50ZXh0LnkgPSBlbGVtZW50X3RleHQoYW5nbGU9MTgwLCBzaXplPTM2KSwKICAgICAgICBheGlzLnRleHQueSA9IGVsZW1lbnRfdGV4dChzaXplPTIwKSwKICAgICAgICBwYW5lbC5iYWNrZ3JvdW5kID0gZWxlbWVudF9yZWN0KGZpbGwgPSBOQSksCiAgICAgICAgcGFuZWwuZ3JpZC5tYWpvciA9IGVsZW1lbnRfbGluZShjb2xvdXIgPSAiZ3JleTgwIikpICsgCiAgeGxhYigiIikrCiAgeWxhYigiIikrCiAgc2NhbGVfeF9jb250aW51b3VzKGxpbWl0cyA9IGMoLTAuMjUsMSksIGJyZWFrcz1jKC0wLjI1LCAwLCAwLjI1LCAwLjUsIDAuNzUsIDEpKSsKICBzY2FsZV95X2Rpc2NyZXRlKGxhYmVscyA9IGZ1bmN0aW9uKHgpIHN0cl93cmFwKHgsIHdpZHRoID0gMTApKSsKICBnZW9tX3ZsaW5lKHhpbnRlcmNlcHQgPSAwLCBjb2xvciA9ICJyZWQiLCBzaXplID0gMSkKICAKcDYgPC0gYXJyYW5nZUdyb2IocDQsIHA1LG5yb3c9MSkKCmdnc2F2ZSgnQm9vdHN0cmFwX1Jhd19WYXJfUGxvdC5qcGcnLCBwbG90ID0gcDYsIGRldmljZSA9ICJqcGVnIiwgcGF0aCA9ICIvVXNlcnMvemV5bmVwZW5rYXZpL0Ryb3Bib3gvUG9sZHJhY2tMYWIvU1JPX1JldGVzdF9BbmFseXNlcy9vdXRwdXQvZmlndXJlcy8iLCB3aWR0aCA9IDM2LCBoZWlnaHQgPSA3MiwgdW5pdHMgPSAiaW4iLCBsaW1pdHNpemUgPSBGQUxTRSwgZHBpPTUwKQoKcm0ocDQsIHA1LCBwNikKYGBgCgpgYGB7ciBlY2hvPUZBTFNFLCBvdXQud2lkdGg9JzEwMCUnfQprbml0cjo6aW5jbHVkZV9ncmFwaGljcygnL1VzZXJzL3pleW5lcGVua2F2aS9Ecm9wYm94L1BvbGRyYWNrTGFiL1NST19SZXRlc3RfQW5hbHlzZXMvb3V0cHV0L2ZpZ3VyZXMvQm9vdHN0cmFwX1Jhd19WYXJfUGxvdC5qcGcnKQpgYGAKCmBgYHtyIG1lc3NhZ2U9RkFMU0UsIHdhcm5pbmc9RkFMU0V9CnA0X3QgPC0gdG1wICU+JQogIGZpbHRlcih0YXNrID09ICd0YXNrJykgJT4lCmdncGxvdChhZXMoeCA9IGZhY3Rvcih0YXNrX2dyb3VwLCBsZXZlbHM9cmV2KHVuaXF1ZSh0YXNrX2dyb3VwKSkpLCB5ID0gaWNjKSkgKyAKICBnZW9tX3Zpb2xpbihmaWxsPScjMDBCRkM0JykrCiAgdGhlbWVfYncoKSArIAogIHRoZW1lKGF4aXMudGV4dC55ID0gZWxlbWVudF90ZXh0KHNpemU9MzApKSsKICB4bGFiKCIiKSsKICB5bGFiKCIiKSsKICBzY2FsZV95X2NvbnRpbnVvdXMobGltaXRzID0gYygtMC4yNSwxKSwgYnJlYWtzPWMoLTAuMjUsIDAsIDAuMjUsIDAuNSwgMC43NSwgMSkpKwogIHNjYWxlX3hfZGlzY3JldGUobGFiZWxzID0gZnVuY3Rpb24oeCkgc3RyX3dyYXAoeCwgd2lkdGggPSAxNSkpKwogIGdlb21faGxpbmUoeWludGVyY2VwdCA9IDAsIGNvbG9yID0gInJlZCIsIHNpemUgPSAxKSsKICBjb29yZF9mbGlwKCkKCnA1X3QgPC0gdG1wICU+JQogIGZpbHRlcih0YXNrID09ICdzdXJ2ZXknKSAlPiUKZ2dwbG90KGFlcyh4ID0gZmFjdG9yKHRhc2tfZ3JvdXAsIGxldmVscz1yZXYodW5pcXVlKHRhc2tfZ3JvdXApKSksIHkgPSBpY2MpKSArIAogIGdlb21fdmlvbGluKGZpbGw9JyNGODc2NkQnKSsKICB0aGVtZV9idygpICsgCiAgdGhlbWUoYXhpcy50ZXh0LnkgPSBlbGVtZW50X3RleHQoc2l6ZT00MykpKwogIHhsYWIoIiIpKwogIHlsYWIoIiIpKwogIHNjYWxlX3lfY29udGludW91cyhsaW1pdHMgPSBjKC0wLjI1LDEpLCBicmVha3M9YygtMC4yNSwgMCwgMC4yNSwgMC41LCAwLjc1LCAxKSkrCiAgc2NhbGVfeF9kaXNjcmV0ZShsYWJlbHMgPSBmdW5jdGlvbih4KSBzdHJfd3JhcCh4LCB3aWR0aCA9IDEwKSkrCiAgZ2VvbV9obGluZSh5aW50ZXJjZXB0ID0gMCwgY29sb3IgPSAicmVkIiwgc2l6ZSA9IDEpKwogIGNvb3JkX2ZsaXAoKQogIApwNl90IDwtIGFycmFuZ2VHcm9iKHA0X3QsIHA1X3QsbnJvdz0xKQoKZ2dzYXZlKCdCb290c3RyYXBfUG9zdGVyX1Bsb3RfdC5qcGcnLCBwbG90ID0gcDZfdCwgZGV2aWNlID0gImpwZWciLCBwYXRoID0gIi4uL291dHB1dC9maWd1cmVzLyIsIHdpZHRoID0gMjcsIGhlaWdodCA9IDQ4LCB1bml0cyA9ICJpbiIsIGxpbWl0c2l6ZSA9IEZBTFNFLCBkcGkgPSAxMDApCmBgYAoKYGBge3IgZWNobz1GQUxTRSwgb3V0LndpZHRoPScxMDAlJ30Ka25pdHI6OmluY2x1ZGVfZ3JhcGhpY3MoJy9Vc2Vycy96ZXluZXBlbmthdmkvRHJvcGJveC9Qb2xkcmFja0xhYi9TUk9fUmV0ZXN0X0FuYWx5c2VzL291dHB1dC9maWd1cmVzL0Jvb3RzdHJhcF9Qb3N0ZXJfUGxvdF90LmpwZycpCmBgYAoKYGBge3IgZWNobz1GQUxTRX0Kcm0ocDRfdCwgcDVfdCwgcDZfdCkKYGBgCgpFeGFtcGxlIG9mIHRoZSBvdmVybGF5aW5nIHByb2NlZHVyZS4KCmBgYHtyfQpwMTwtIHRtcCAlPiUKICBmaWx0ZXIoZ3JlcGwoJ3NlbGVjdGlvbl9vcHRpbWl6YXRpb24nLCBkdikpICU+JQpnZ3Bsb3QoYWVzKHggPSB2YXIsIHkgPSBpY2MpKSArIAogIGdlb21fdmlvbGluKGZpbGw9JyNGODc2NkQnKSsKICB0aGVtZV9idygpICsgCiAgdGhlbWUoYXhpcy50ZXh0LnkgPSBlbGVtZW50X3RleHQoc2l6ZT0zMCkpKwogIHhsYWIoIiIpKwogIHlsYWIoIiIpKwogIHNjYWxlX3lfY29udGludW91cyhsaW1pdHMgPSBjKC0wLjI1LDEpLCBicmVha3M9YygtMC4yNSwgMCwgMC4yNSwgMC41LCAwLjc1LCAxKSkrCiAgc2NhbGVfeF9kaXNjcmV0ZShsYWJlbHMgPSBmdW5jdGlvbih4KSBzdHJfd3JhcCh4LCB3aWR0aCA9IDE1KSkrCiAgZ2VvbV9obGluZSh5aW50ZXJjZXB0ID0gMCwgY29sb3IgPSAicmVkIiwgc2l6ZSA9IDEpKwogIGNvb3JkX2ZsaXAoKQoKcDI8LSB0bXAgJT4lCiAgZmlsdGVyKGdyZXBsKCdzZWxlY3Rpb25fb3B0aW1pemF0aW9uJywgZHYpKSAlPiUKZ2dwbG90KGFlcyh4ID0gZmFjdG9yKHRhc2tfZ3JvdXAsIGxldmVscz1yZXYodW5pcXVlKHRhc2tfZ3JvdXApKSksIHkgPSBpY2MsIGdyb3VwPWR2KSkgKyAKICBnZW9tX3Zpb2xpbihmaWxsPScjRjg3NjZEJywgcG9zaXRpb24gPSAnaWRlbnRpdHknKSsKICB0aGVtZV9idygpICsgCiAgdGhlbWUoYXhpcy50ZXh0LnkgPSBlbGVtZW50X3RleHQoc2l6ZT0zMCkpKwogIHhsYWIoIiIpKwogIHlsYWIoIiIpKwogIHNjYWxlX3lfY29udGludW91cyhsaW1pdHMgPSBjKC0wLjI1LDEpLCBicmVha3M9YygtMC4yNSwgMCwgMC4yNSwgMC41LCAwLjc1LCAxKSkrCiAgc2NhbGVfeF9kaXNjcmV0ZShsYWJlbHMgPSBmdW5jdGlvbih4KSBzdHJfd3JhcCh4LCB3aWR0aCA9IDE1KSkrCiAgZ2VvbV9obGluZSh5aW50ZXJjZXB0ID0gMCwgY29sb3IgPSAicmVkIiwgc2l6ZSA9IDEpKwogIGNvb3JkX2ZsaXAoKQoKcDM8LSB0bXAgJT4lCiAgZmlsdGVyKGdyZXBsKCdzZWxlY3Rpb25fb3B0aW1pemF0aW9uJywgZHYpKSAlPiUKZ2dwbG90KGFlcyh4ID0gZmFjdG9yKHRhc2tfZ3JvdXAsIGxldmVscz1yZXYodW5pcXVlKHRhc2tfZ3JvdXApKSksIHkgPSBpY2MpKSArIAogIGdlb21fdmlvbGluKGZpbGw9JyNGODc2NkQnLCBwb3NpdGlvbiA9ICdpZGVudGl0eScpKwogIHRoZW1lX2J3KCkgKyAKICB0aGVtZShheGlzLnRleHQueSA9IGVsZW1lbnRfdGV4dChzaXplPTMwKSkrCiAgeGxhYigiIikrCiAgeWxhYigiIikrCiAgc2NhbGVfeV9jb250aW51b3VzKGxpbWl0cyA9IGMoLTAuMjUsMSksIGJyZWFrcz1jKC0wLjI1LCAwLCAwLjI1LCAwLjUsIDAuNzUsIDEpKSsKICBzY2FsZV94X2Rpc2NyZXRlKGxhYmVscyA9IGZ1bmN0aW9uKHgpIHN0cl93cmFwKHgsIHdpZHRoID0gMTUpKSsKICBnZW9tX2hsaW5lKHlpbnRlcmNlcHQgPSAwLCBjb2xvciA9ICJyZWQiLCBzaXplID0gMSkrCiAgY29vcmRfZmxpcCgpCgpwNCA8LSBhcnJhbmdlR3JvYihwMSxwMiwgcDMsbnJvdz0xKQoKZ2dzYXZlKCdCb290c3RyYXBfRXhhbXBsZV9QbG90X3QuanBnJywgcGxvdCA9IHA0LCBkZXZpY2UgPSAianBlZyIsIHBhdGggPSAiLi4vb3V0cHV0L2ZpZ3VyZXMvIiwgd2lkdGggPSA0MSwgaGVpZ2h0ID0gNSwgdW5pdHMgPSAiaW4iLCBsaW1pdHNpemUgPSBGQUxTRSwgZHBpID0gMTAwKQpgYGAKCmBgYHtyIGVjaG89RkFMU0UsIG91dC53aWR0aD0nMTAwJSd9CmtuaXRyOjppbmNsdWRlX2dyYXBoaWNzKCcvVXNlcnMvemV5bmVwZW5rYXZpL0Ryb3Bib3gvUG9sZHJhY2tMYWIvU1JPX1JldGVzdF9BbmFseXNlcy9vdXRwdXQvZmlndXJlcy9Cb290c3RyYXBfRXhhbXBsZV9QbG90X3QuanBnJykKYGBgCgpgYGB7cn0Kcm0ocDEsIHAyLCBwMywgcDQpCmBgYAoKIyMgU3VydmV5IHZzIFRhc2tzCgpDb21wYXJpc29uIG9mIHN1cnZleSBtZWFzdXJlcyB0byBjb2duaXRpdmUgdGFzayBtZWFzdXJlcyBpbiB0aGUgYm9vdHN0cmFwcGVkIHJlc3VsdHMuIE11bHRpbGV2ZWwgbW9kZWwgd2l0aCByYW5kb20gaW50ZXJjZXB0cyBmb3IgZWFjaCBtZWFzdXJlIGFuZCBmaXhlZCBlZmZlY3Qgb2Ygc3VydmV5IHZlcnN1cyBjb2duaXRpdmUgbWVhc3VyZS4gCgpgYGB7cn0KYm9vdF9kZiA9IGJvb3RfZGYgJT4lCiAgICBtdXRhdGUodGFzayA9IGlmZWxzZShncmVwbCgic3VydmV5IixkdiksICJzdXJ2ZXkiLCJ0YXNrIiksCiAgICAgICAgICAgdGFzayA9IGlmZWxzZShncmVwbCgiaG9sdCIsZHYpLCAidGFzayIsIHRhc2spKQoKYm9vdF9kZiAlPiUKICBncm91cF9ieSh0YXNrKSAlPiUKICBzdW1tYXJpc2UoaWNjX21lZGlhbiA9IHF1YW50aWxlKGljYywgcHJvYnMgPSAwLjUpLAogICAgICAgICAgICBpY2NfMi41ID0gcXVhbnRpbGUoaWNjLCBwcm9icyA9IDAuMDI1KSwKICAgICAgICAgICAgaWNjXzk3LjUgPSBxdWFudGlsZShpY2MsIHByb2JzID0gMC45NzUpLAogICAgICAgICAgICBzcGVhcm1hbl9tZWRpYW4gPSBxdWFudGlsZShzcGVhcm1hbiwgcHJvYnMgPSAwLjUpLAogICAgICAgICAgICBzcGVhcm1hbl8yLjUgPSBxdWFudGlsZShzcGVhcm1hbiwgcHJvYnMgPSAwLjAyNSksCiAgICAgICAgICAgIHNwZWFybWFuXzk3LjUgPSBxdWFudGlsZShzcGVhcm1hbiwgcHJvYnMgPSAwLjk3NSksCiAgICAgICAgICAgIG51bV92YXJzID0gbigpLzEwMDApICU+JQogIGRhdGF0YWJsZSgpICU+JQogIGZvcm1hdFJvdW5kKGNvbHVtbnM9YygnaWNjX21lZGlhbicsICdpY2NfMi41JywgJ2ljY185Ny41JywgJ3NwZWFybWFuX21lZGlhbicsICdzcGVhcm1hbl8yLjUnLCAnc3BlYXJtYW5fOTcuNScpLCBkaWdpdHM9MykKYGBgCgoKYGBge3J9CnN1bW1hcnkobG1lclRlc3Q6OmxtZXIoaWNjIH4gIHRhc2sgKyAoMXxkdiksIGJvb3RfZGYpKQpgYGAKCiMjIyBWYXJpYW5jZSBicmVha2Rvd24KClRoZSBxdWFudGlhdGl2ZSBleHBsYW5hdGlvbiBmb3IgdGhlIGRpZmZlcmVuY2UgaW4gcmVsaWFiaWxpdHkgZXN0aW1hdGVzIGJldHdlZW4gc3VydmV5cyBhbmQgdGFza3MsIGFzIHJlY2VudGx5IGRldGFpbGVkIGJ5IEhlZGdlIGV0IGFsLiAoMjAxNyksIGxpZXMgaW4gdGhlIGRpZmZlcmVuY2UgaW4gc291cmNlcyBvZiB2YXJpYW5jZSBiZXR3ZWVuIHRoZXNlIG1lYXN1cmVzLiBTcGVjaWZpY2FsbHksIHRoZSBJQ0MgaXMgY2FsY3VsYXRlZCBhcyB0aGUgcmF0aW8gb2YgdmFyaWFuY2UgYmV0d2VlbiBzdWJqZWN0cyB2YXJpYW5jZSB0byBhbGwgc291cmNlcyBvZiB2YXJpYW5jZS4gVGh1cywgbWVhc3VyZXMgd2l0aCBoaWdoIGJldHdlZW4gc3ViamVjdHMgdmFyaWFuY2Ugd291bGQgaGF2ZSBoaWdoIHRlc3QtcmV0ZXN0IHJlbGlhYmlsaXR5LiBJbnR1aXRpdmVseSwgbWVhc3VyZXMgd2l0aCBoaWdoIGJldHdlZW4gc3ViamVjdHMgdmFyaWFuY2UgYXJlIGFsc28gYmV0dGVyIHN1aXRlZCBmb3IgaW5kaXZpZHVhbCBkaWZmZXJlbmNlIGFuYWx5c2VzIGFzIHRoZXkgd291bGQgY2FwdHVyZSB0aGUgZGlmZmVyZW5jZXMgYmV0d2VlbiB0aGUgc3ViamVjdHMgaW4gYSBzYW1wbGUuCgpIZXJlIHdlIGZpcnN0IHBsb3QgdGhlIHBlcmNlbnRhZ2Ugb2YgdmFyaWFuY2UgZXhwbGFpbmVkIGJ5IHRoZSB0aHJlZSBzb3VyY2VzIG9mIHZhcmlhbmNlIGZvciB0aGUgcG9pbnQgZXN0aW1hdGVzIG9mIG1lYXN1cmUgcmVsaWFiaWxpdGllcy4gVGhlIHBsb3Qgb25seSBpbmNsdWRlcyByYXcgbWVhc3VyZXMgKG5vIERETSBwYXJhbWV0ZXJzKSBhbmQgdGhlIG1lYXN1cmVzIGFyZSByYW5rZWQgYnkgcGVyY2VudGFnZSBvZiBiZXR3ZWVuIHN1YmplY3QgdmFyaWFiaWxpdHkgZm9yIGVhY2ggdGFzay9zdXJ2ZXkgKGkuZS4gdGhlIGJlc3QgdG8gd29yc3QgaW5kaXZpZHVhbCBkaWZmZXJlbmNlIG1lYXN1cmUgZm9yIGVhY2ggdGFzay9zdXJ2ZXkpLiBUaGVuIHdlIGNvbXBhcmUgc3RhdGlzdGljYWxseSB3aGV0aGVyIHRoZSBwZXJjZW50YWdlIG9mIHZhcmlhbmNlIGV4cGxhaW5lZCBieSB0aGVzZSBzb3VyY2VzIGRpZmZlciBiZXR3ZWVuIHRhc2tzIGFuZCBzdXJ2ZXlzLgoKYGBge3Igd2FybmluZz1GQUxTRSwgbWVzc2FnZT1GQUxTRX0KdG1wID0gcmVsX2RmICU+JQogIG11dGF0ZSh2YXJfc3Vic19wY3QgPSB2YXJfc3Vicy8odmFyX3N1YnMrdmFyX2luZCt2YXJfcmVzaWQpKjEwMCwKICAgICAgICAgdmFyX2luZF9wY3QgPSB2YXJfaW5kLyh2YXJfc3Vicyt2YXJfaW5kK3Zhcl9yZXNpZCkqMTAwLCAKICAgICAgICAgdmFyX3Jlc2lkX3BjdCA9IHZhcl9yZXNpZC8odmFyX3N1YnMrdmFyX2luZCt2YXJfcmVzaWQpKjEwMCkgJT4lCiAgc2VsZWN0KGR2LCB0YXNrLCB2YXJfc3Vic19wY3QsIHZhcl9pbmRfcGN0LCB2YXJfcmVzaWRfcGN0KSAlPiUKICBtdXRhdGUoZHYgPSBmYWN0b3IoZHYsIGxldmVscyA9IGR2W29yZGVyKHRhc2spXSkpICU+JQogIHNlcGFyYXRlKGR2LCBjKCJ0YXNrX2dyb3VwIiwgInZhciIpLCBzZXA9IlxcLiIscmVtb3ZlPUZBTFNFLGV4dHJhPSJtZXJnZSIpICU+JQogIG11dGF0ZSh0YXNrX2dyb3VwID0gZmFjdG9yKHRhc2tfZ3JvdXAsIGxldmVscyA9IHRhc2tfZ3JvdXBbb3JkZXIodGFzayldKSkgJT4lCiAgYXJyYW5nZSh0YXNrX2dyb3VwLCB2YXJfc3Vic19wY3QpICU+JQogIG11dGF0ZShyYW5rID0gcm93X251bWJlcigpKSAlPiUKICBhcnJhbmdlKHRhc2ssIHRhc2tfZ3JvdXAsIHJhbmspICU+JQogIGdhdGhlcihrZXksIHZhbHVlLCAtZHYsIC10YXNrX2dyb3VwLCAtdmFyLCAtdGFzaywgLXJhbmspICU+JQogIHVuZ3JvdXAoKSU+JQogIG11dGF0ZSh0YXNrX2dyb3VwID0gZ3N1YigiXyIsICIgIiwgdGFza19ncm91cCksCiAgICAgICAgIHZhciA9IGdzdWIoIl8iLCAiICIsIHZhcikpICU+JQogIG11dGF0ZSh0YXNrX2dyb3VwID0gaWZlbHNlKHRhc2tfZ3JvdXAgPT0gInBzeWNob2xvZ2ljYWwgcmVmcmFjdG9yeSBwZXJpb2QgdHdvIGNob2ljZXMiLCAicHN5Y2hvbG9naWNhbCByZWZyYWN0b3J5IHBlcmlvZCIsIGlmZWxzZSh0YXNrX2dyb3VwID09ICJhbmdsaW5nIHJpc2sgdGFzayBhbHdheXMgc3VubnkiLCAiYW5nbGluZyByaXNrIHRhc2siLHRhc2tfZ3JvdXApKSkgJT4lCiAgbXV0YXRlKHRhc2tfZ3JvdXAgPSBnc3ViKCJzdXJ2ZXkiLCAiIiwgdGFza19ncm91cCkpICU+JQogIGZpbHRlcih0YXNrPT0idGFzayIsCiAgICAgICAgICFncmVwbCgiRVp8aGRkbSIsIGR2KSklPiUKICBhcnJhbmdlKHRhc2tfZ3JvdXAsIHJhbmspCmxhYmVscyA9IHRtcCAlPiUKICBkaXN0aW5jdChkdiwgLmtlZXBfYWxsPVQpCgpwMSA8LSB0bXAgJT4lCiAgZ2dwbG90KGFlcyh4PWZhY3RvcihyYW5rKSwgeT12YWx1ZSwgZmlsbD1mYWN0b3Ioa2V5LCBsZXZlbHMgPSBjKCJ2YXJfcmVzaWRfcGN0IiwgInZhcl9pbmRfcGN0IiwgInZhcl9zdWJzX3BjdCIpKSkpKwogIGdlb21fYmFyKHN0YXQ9J2lkZW50aXR5JywgYWxwaGEgPSAwLjc1LCBjb2xvcj0nIzAwQkZDNCcpKwogIHNjYWxlX3hfZGlzY3JldGUoYnJlYWtzID0gbGFiZWxzJHJhbmssCiAgICAgICAgICAgICAgICAgICAgICAgbGFiZWxzID0gbGFiZWxzJHZhcikrCiAgY29vcmRfZmxpcCgpKwogIGZhY2V0X2dyaWQodGFza19ncm91cH4uLCBzd2l0Y2ggPSAieSIsIHNjYWxlcyA9ICJmcmVlX3kiLCBzcGFjZSA9ICJmcmVlX3kiKSArCiAgdGhlbWUocGFuZWwuc3BhY2luZyA9IHVuaXQoMC41LCAibGluZXMiKSwgCiAgICAgICAgc3RyaXAucGxhY2VtZW50ID0gIm91dHNpZGUiLAogICAgICAgIHN0cmlwLnRleHQueSA9IGVsZW1lbnRfdGV4dChhbmdsZT0xODApLAogICAgICAgIHBhbmVsLmJhY2tncm91bmQgPSBlbGVtZW50X3JlY3QoZmlsbCA9IE5BKSwKICAgICAgICBwYW5lbC5ncmlkLm1ham9yID0gZWxlbWVudF9saW5lKGNvbG91ciA9ICJncmV5ODUiKSwKICAgICAgICBsZWdlbmQucG9zaXRpb24gPSAnYm90dG9tJykrCiAgdGhlbWUobGVnZW5kLnRpdGxlID0gZWxlbWVudF9ibGFuaygpKSsKICBzY2FsZV9maWxsX21hbnVhbChicmVha3MgPSBjKCJ2YXJfc3Vic19wY3QiLCAidmFyX2luZF9wY3QiLCAidmFyX3Jlc2lkX3BjdCIpLAogICAgICAgICAgICAgICAgICAgICAgbGFiZWxzID0gYygiVmFyaWFuY2UgYmV0d2VlbiBpbmRpdmlkdWFscyIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICJWYXJpYW5jZSBiZXR3ZWVuIHNlc3Npb25zIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIkVycm9yIHZhcmlhbmNlIiksCiAgICAgICAgICAgICAgICAgIHZhbHVlcz1jKCJncmV5NjUiLCAiZ3JleTQ1IiwgImdyZXkyNSIpKSsKICB5bGFiKCIiKSsKICB4bGFiKCIiKQoKdG1wID0gcmVsX2RmICU+JQogIG11dGF0ZSh2YXJfc3Vic19wY3QgPSB2YXJfc3Vicy8odmFyX3N1YnMrdmFyX2luZCt2YXJfcmVzaWQpKjEwMCwKICAgICAgICAgdmFyX2luZF9wY3QgPSB2YXJfaW5kLyh2YXJfc3Vicyt2YXJfaW5kK3Zhcl9yZXNpZCkqMTAwLCAKICAgICAgICAgdmFyX3Jlc2lkX3BjdCA9IHZhcl9yZXNpZC8odmFyX3N1YnMrdmFyX2luZCt2YXJfcmVzaWQpKjEwMCkgJT4lCiAgc2VsZWN0KGR2LCB0YXNrLCB2YXJfc3Vic19wY3QsIHZhcl9pbmRfcGN0LCB2YXJfcmVzaWRfcGN0KSAlPiUKICBtdXRhdGUoZHYgPSBmYWN0b3IoZHYsIGxldmVscyA9IGR2W29yZGVyKHRhc2spXSkpICU+JQogIHNlcGFyYXRlKGR2LCBjKCJ0YXNrX2dyb3VwIiwgInZhciIpLCBzZXA9IlxcLiIscmVtb3ZlPUZBTFNFLGV4dHJhPSJtZXJnZSIpICU+JQogIG11dGF0ZSh0YXNrX2dyb3VwID0gZmFjdG9yKHRhc2tfZ3JvdXAsIGxldmVscyA9IHRhc2tfZ3JvdXBbb3JkZXIodGFzayldKSkgJT4lCiAgYXJyYW5nZSh0YXNrX2dyb3VwLCB2YXJfc3Vic19wY3QpICU+JQogIG11dGF0ZShyYW5rID0gcm93X251bWJlcigpKSAlPiUKICBhcnJhbmdlKHRhc2ssIHRhc2tfZ3JvdXAsIHJhbmspICU+JQogIGdhdGhlcihrZXksIHZhbHVlLCAtZHYsIC10YXNrX2dyb3VwLCAtdmFyLCAtdGFzaywgLXJhbmspICU+JQogIHVuZ3JvdXAoKSU+JQogIG11dGF0ZSh0YXNrX2dyb3VwID0gZ3N1YigiXyIsICIgIiwgdGFza19ncm91cCksCiAgICAgICAgIHZhciA9IGdzdWIoIl8iLCAiICIsIHZhcikpICU+JQogIG11dGF0ZSh0YXNrX2dyb3VwID0gaWZlbHNlKHRhc2tfZ3JvdXAgPT0gInBzeWNob2xvZ2ljYWwgcmVmcmFjdG9yeSBwZXJpb2QgdHdvIGNob2ljZXMiLCAicHN5Y2hvbG9naWNhbCByZWZyYWN0b3J5IHBlcmlvZCIsIGlmZWxzZSh0YXNrX2dyb3VwID09ICJhbmdsaW5nIHJpc2sgdGFzayBhbHdheXMgc3VubnkiLCAiYW5nbGluZyByaXNrIHRhc2siLHRhc2tfZ3JvdXApKSkgJT4lCiAgbXV0YXRlKHRhc2tfZ3JvdXAgPSBnc3ViKCJzdXJ2ZXkiLCAiIiwgdGFza19ncm91cCkpICU+JQogIGZpbHRlcih0YXNrPT0ic3VydmV5IiklPiUKICBhcnJhbmdlKHRhc2tfZ3JvdXAsIHJhbmspCmxhYmVscyA9IHRtcCAlPiUKICBkaXN0aW5jdChkdiwgLmtlZXBfYWxsPVQpCgpwMiA8LSB0bXAgJT4lCiAgZ2dwbG90KGFlcyh4PWZhY3RvcihyYW5rKSwgeT12YWx1ZSwgZmlsbD1mYWN0b3Ioa2V5LCBsZXZlbHMgPSBjKCJ2YXJfcmVzaWRfcGN0IiwgInZhcl9pbmRfcGN0IiwgInZhcl9zdWJzX3BjdCIpKSkpKwogIGdlb21fYmFyKHN0YXQ9J2lkZW50aXR5JywgYWxwaGEgPSAwLjc1KSsKICBnZW9tX2JhcihzdGF0PSdpZGVudGl0eScsIGNvbG9yPScjRjg3NjZEJywgc2hvdy5sZWdlbmQ9RkFMU0UpKwogIHNjYWxlX3hfZGlzY3JldGUoYnJlYWtzID0gbGFiZWxzJHJhbmssCiAgICAgICAgICAgICAgICAgICAgICAgbGFiZWxzID0gbGFiZWxzJHZhcikrCiAgY29vcmRfZmxpcCgpKwogIGZhY2V0X2dyaWQodGFza19ncm91cH4uLCBzd2l0Y2ggPSAieSIsIHNjYWxlcyA9ICJmcmVlX3kiLCBzcGFjZSA9ICJmcmVlX3kiKSArCiAgdGhlbWUocGFuZWwuc3BhY2luZyA9IHVuaXQoMC41LCAibGluZXMiKSwgCiAgICAgICAgc3RyaXAucGxhY2VtZW50ID0gIm91dHNpZGUiLAogICAgICAgIHN0cmlwLnRleHQueSA9IGVsZW1lbnRfdGV4dChhbmdsZT0xODApLAogICAgICAgIHBhbmVsLmJhY2tncm91bmQgPSBlbGVtZW50X3JlY3QoZmlsbCA9IE5BKSwKICAgICAgICBwYW5lbC5ncmlkLm1ham9yID0gZWxlbWVudF9saW5lKGNvbG91ciA9ICJncmV5ODUiKSwKICAgICAgICBsZWdlbmQucG9zaXRpb24gPSAnYm90dG9tJykrCiAgdGhlbWUobGVnZW5kLnRpdGxlID0gZWxlbWVudF9ibGFuaygpKSsKICBzY2FsZV9maWxsX21hbnVhbChicmVha3MgPSBjKCJ2YXJfc3Vic19wY3QiLCAidmFyX2luZF9wY3QiLCAidmFyX3Jlc2lkX3BjdCIpLAogICAgICAgICAgICAgICAgICAgICAgbGFiZWxzID0gYygiVmFyaWFuY2UgYmV0d2VlbiBpbmRpdmlkdWFscyIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICJWYXJpYW5jZSBiZXR3ZWVuIHNlc3Npb25zIiwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIkVycm9yIHZhcmlhbmNlIiksCiAgICAgICAgICAgICAgICAgIHZhbHVlcz1jKCJncmV5NjUiLCAiZ3JleTQ1IiwgImdyZXkyNSIpKSsKICB5bGFiKCIiKSsKICB4bGFiKCIiKQoKbXlsZWdlbmQ8LWdfbGVnZW5kKHAyKQoKcDMgPC0gYXJyYW5nZUdyb2IoYXJyYW5nZUdyb2IocDEgK3RoZW1lKGxlZ2VuZC5wb3NpdGlvbj0ibm9uZSIpLAogICAgICAgICAgICAgICAgICAgICAgICAgcDIgKyB0aGVtZShsZWdlbmQucG9zaXRpb249Im5vbmUiKSwKICAgICAgICAgICAgICAgICAgICAgICAgIG5yb3c9MSksCiAgICAgICAgICAgICBteWxlZ2VuZCwgbnJvdz0yLGhlaWdodHM9YygxMCwgMSkpCgpnZ3NhdmUoJ1ZhcmlhbmNlX0JyZWFrZG93bl9QbG90LmpwZycsIHBsb3QgPSBwMywgZGV2aWNlID0gImpwZWciLCBwYXRoID0gIi9Vc2Vycy96ZXluZXBlbmthdmkvRHJvcGJveC9Qb2xkcmFja0xhYi9TUk9fUmV0ZXN0X0FuYWx5c2VzL291dHB1dC9maWd1cmVzIiwgd2lkdGggPSAyNCwgaGVpZ2h0ID0gMjAsIHVuaXRzID0gImluIiwgZHBpID0gMTAwKQpybSh0bXAsIGxhYmVscywgcDEsIHAyICwgcDMpCmBgYAoKYGBge3IgZWNobz1GQUxTRSwgb3V0LndpZHRoPScxMDAlJ30Ka25pdHI6OmluY2x1ZGVfZ3JhcGhpY3MoJy9Vc2Vycy96ZXluZXBlbmthdmkvRHJvcGJveC9Qb2xkcmFja0xhYi9TUk9fUmV0ZXN0X0FuYWx5c2VzL291dHB1dC9maWd1cmVzL1ZhcmlhbmNlX0JyZWFrZG93bl9QbG90LmpwZycpCmBgYAoKQ29tcGFyaW5nIHR5cGVzIG9mIHZhcmlhbmNlIGZvciBzdXJ2ZXkgdnMgdGFzayBtZWFzdXJlczogU3VydmV5IG1lYXN1cmVzIGhhdmUgaGlnaGVyIGJldHdlZW4gc3ViamVjdCB2YXJpYWJpbGl0eSAgCgpOb3RlOiBUaGlzIGFuYWx5c2lzIGluY2x1ZGVzIERETSB2YXJpYWJsZXMgdG9vLgoKUnVubmluZyBzZXBhcmF0ZSBtb2RlbHMgZm9yIGRpZmZlcmVudCBzb3VyY2VzIG9mIHZhcmlhbmNlIGJlY2F1c2UgaW50ZXJhY3RpdmUgbW9kZWwgd2l0aCB2YXJpYW5jZSB0eXBlKnRhc2sgc2VlbWVkIHRvbyBjb21wbGljYXRlZC4KCkZpcnN0IHdlIGZpbmQgdGhhdCB0YXNrIG1lYXN1cmVzIGhhdmUgYSBzbWFsbGVyIHBlcmNlbnRhZ2Ugb2YgdGhlaXIgb3ZlcmFsbCB2YXJpYW5jZSBleHBsYWluZWQgYnkgdmFyaWFiaWxpdHkgYmV0d2VlbiBzdWJqZWN0cyBjb21wYXJlZCB0byBzdXJ2ZXkgbWVhc3VyZXMuCgpgYGB7cn0KdG1wID0gYm9vdF9kZiAlPiUKICBtdXRhdGUodmFyX3N1YnNfcGN0ID0gdmFyX3N1YnMvKHZhcl9zdWJzK3Zhcl9pbmQrdmFyX3Jlc2lkKSoxMDAsCiAgICAgICAgIHZhcl9pbmRfcGN0ID0gdmFyX2luZC8odmFyX3N1YnMrdmFyX2luZCt2YXJfcmVzaWQpKjEwMCwgCiAgICAgICAgIHZhcl9yZXNpZF9wY3QgPSB2YXJfcmVzaWQvKHZhcl9zdWJzK3Zhcl9pbmQrdmFyX3Jlc2lkKSoxMDApICU+JQogIHNlbGVjdChkdiwgdGFzaywgdmFyX3N1YnNfcGN0LCB2YXJfaW5kX3BjdCwgdmFyX3Jlc2lkX3BjdCkgCgpzdW1tYXJ5KGxtZXJUZXN0OjpsbWVyKHZhcl9zdWJzX3BjdH50YXNrKygxfGR2KSx0bXAlPiVzZWxlY3QoLXZhcl9pbmRfcGN0LC12YXJfcmVzaWRfcGN0KSkpCmBgYAoKV2UgYWxzbyBmaW5kIHRoYXQgYSBzaWduaWZpY2FudGx5IGxhcmdlciBwZXJjZW50YWdlIG9mIHRoZWlyIHZhcmlhbmNlIGlzIGV4cGxhaW5lZCBieSBiZXR3ZWVuIHNlc3Npb24gdmFyaWFiaWxpdHkuIExhcmdlciBiZXR3ZWVuIHNlc3Npb24gdmFyaWFiaWxpdHkgc3VnZ2VzdHMgc3lzdGVtYXRpYyBkaWZmZXJlbmNlcyBiZXR3ZWVuIHRoZSB0d28gc2Vzc2lvbnMuIFN1Y2ggc3lzdGVtYXRpYyBlZmZlY3RzIGNhbiBiZSBkdWUgdG8gZS5nLiBsZWFybmluZyBlZmZlY3RzIGFzIGV4cGxvcmVkIGxhdGVyLgoKYGBge3J9CnN1bW1hcnkobG1lclRlc3Q6OmxtZXIodmFyX2luZF9wY3R+dGFzaysoMXxkdiksdG1wJT4lc2VsZWN0KC12YXJfc3Vic19wY3QsLXZhcl9yZXNpZF9wY3QpKSkKYGBgCgpgYGB7cn0Kc3VtbWFyeShsbWVyVGVzdDo6bG1lcih2YXJfcmVzaWRfcGN0fnRhc2srKDF8ZHYpLHRtcCU+JXNlbGVjdCgtdmFyX3N1YnNfcGN0LC12YXJfaW5kX3BjdCkpKQpgYGAKCmBgYHtyfQp0bXBfc2F2ZSA9IHRtcCU+JQogIGdhdGhlcihrZXksIHZhbHVlLCAtZHYsIC10YXNrKSAlPiUKICBncm91cF9ieSh0YXNrLCBrZXkpICU+JQogIHN1bW1hcmlzZShtZWRpYW4gPSBtZWRpYW4odmFsdWUpLAogICAgICAgICAgICBzZCA9IHNkKHZhbHVlKSkgJT4lCiAgbXV0YXRlKGtleSA9IGlmZWxzZShrZXkgPT0gJ3Zhcl9pbmRfcGN0JywgJ0JldHdlZW4gc2Vzc2lvbiB2YXJpYW5jZScsIGlmZWxzZShrZXkgPT0gJ3Zhcl9zdWJzX3BjdCcsICdCZXR3ZWVuIHN1YmplY3RzIHZhcmlhbmNlJywgaWZlbHNlKGtleSA9PSAndmFyX3Jlc2lkX3BjdCcsICdSZXNpZHVhbCB2YXJpYW5jZScsTkEpKSkpICU+JQogIHJlbmFtZShNZWRpYW4gPSBtZWRpYW4sIFNEID0gc2QpICU+JQogIGFycmFuZ2UodGFzaywga2V5KQoKdG1wX3NhdmUKYGBgCgpgYGB7cn0Kc2p0LmRmKHRtcF9zYXZlICU+JSBtdXRhdGVfaWYoaXMubnVtZXJpYywgZnVucyhyb3VuZCguLCAzKSkpLCBkZXNjcmliZT1GLCBoaWRlLnByb2dyZXNzID0gVFJVRSwgc2hvdy5yb3duYW1lcyA9IEZBTFNFLCBmaWxlID0gIi9Vc2Vycy96ZXluZXBlbmthdmkvRHJvcGJveC9Qb2xkcmFja0xhYi9TUk9fUmV0ZXN0X0FuYWx5c2VzL291dHB1dC90YWJsZXMvdmFyX2JyZWFrZG93bl9zdW1tYXJ5LmRvYyIpCmBgYAoKU3VtbWFyaXppbmcgZm9yIGNsZWFyZXIgcHJlc2VudGF0aW9uLiBUaGlzIGdyYXBoIGlzIGN1cnJlbnRseSB1c2luZyB0aGUgYm9vdHN0cmFwcGVkIHJlbGlhYmlsaXRpZXMgYW5kIGlzIHRoZXJlZm9yZSBtZXNzaWVyIHRoYW4gaWYganVzdCB1c2luZyB0aGUgcG9pbnQgZXN0aW1hdGVzLgoKYGBge3J9CnRtcCAlPiUKICBnYXRoZXIoa2V5LCB2YWx1ZSwgLWR2LCAtdGFzaykgJT4lCiAgZ3JvdXBfYnkodGFzaywga2V5KSAlPiUKICBzdW1tYXJpc2UobWVhbl9wY3QgPSBtZWFuKHZhbHVlKSwKICAgICAgICAgICAgc2RfcGN0ID0gc2QodmFsdWUpLAogICAgICAgICAgICBuID0gbigpKSAlPiUKICBtdXRhdGUoY3ZsID0gcXQoMC4wMjUsIG4tMSksCiAgICAgICAgIGN2dSA9IHF0KDAuOTc1LCBuLTEpLAogICAgICAgICBjaWwgPSBtZWFuX3BjdCsoc2RfcGN0KmN2bCkvc3FydChuKSwKICAgICAgICAgY2l1ID0gbWVhbl9wY3QrKHNkX3BjdCpjdnUpL3NxcnQobiksCiAgICAgICAgIHNlbV9wY3QgPSBzZF9wY3Qvc3FydChuKSkgJT4lCiAgZ2dwbG90KGFlcyhmYWN0b3Ioa2V5LCBsZXZlbHMgPSBjKCJ2YXJfc3Vic19wY3QiLCAidmFyX2luZF9wY3QiLCAidmFyX3Jlc2lkX3BjdCIpLAogICAgICAgICAgICAgICAgICAgIGxhYmVscyA9IGMoIlZhcmlhbmNlIGJldHdlZW4gaW5kaXZpZHVhbHMiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIlZhcmlhbmNlIGJldHdlZW4gc2Vzc2lvbnMiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIkVycm9yIHZhcmlhbmNlIikpLCBtZWFuX3BjdCwgY29sb3I9dGFzaykpKwogIGdlb21fcG9pbnQocG9zaXRpb249cG9zaXRpb25fZG9kZ2Uod2lkdGggPSAwLjI1KSwgc2l6ZSA9IDQpKwogICMgZ2VvbV9lcnJvcmJhcihhZXMoeW1pbj1tZWFuX3BjdC1zZF9wY3QsIHltYXg9bWVhbl9wY3Qrc2RfcGN0KSwgcG9zaXRpb249cG9zaXRpb25fZG9kZ2Uod2lkdGggPSAwLjI1KSwgd2lkdGg9MCwgc2l6ZT0yKSsKICBnZW9tX2Vycm9yYmFyKGFlcyh5bWluPWNpbCwgeW1heD1jaXUpLCBwb3NpdGlvbj1wb3NpdGlvbl9kb2RnZSh3aWR0aCA9IDAuMjUpLCB3aWR0aD0wLjI1LCBzaXplPTIpKwogIHRoZW1lX2J3KCkrCiAgeGxhYignJykrCiAgeWxhYignUGVyY2VudCcpKwogIHRoZW1lKGxlZ2VuZC50aXRsZSA9IGVsZW1lbnRfYmxhbmsoKSwKICAgICAgICBsZWdlbmQudGV4dCA9IGVsZW1lbnRfdGV4dChzaXplPTE0KSwKICAgICAgICBsZWdlbmQucG9zaXRpb24gPSAnYm90dG9tJywKICAgICAgICBheGlzLnRleHQgPSBlbGVtZW50X3RleHQoc2l6ZT0xMiksCiAgICAgICAgYXhpcy50aXRsZS55ID0gZWxlbWVudF90ZXh0KHNpemU9MTIpKSsKICBzY2FsZV94X2Rpc2NyZXRlKGxhYmVscyA9IGZ1bmN0aW9uKHgpIHN0cl93cmFwKHgsIHdpZHRoID0gMTUpKQoKZ2dzYXZlKCdWYXJpYW5jZV9CcmVha2Rvd25fRG90UGxvdC5qcGcnLCBkZXZpY2UgPSAianBlZyIsIHBhdGggPSAiL1VzZXJzL3pleW5lcGVua2F2aS9Ecm9wYm94L1BvbGRyYWNrTGFiL1NST19SZXRlc3RfQW5hbHlzZXMvb3V0cHV0L2ZpZ3VyZXMiLCB3aWR0aCA9IDYsIGhlaWdodCA9IDUsIHVuaXRzID0gImluIiwgZHBpID0gMTAwKQpgYGAKCiMjIFN5c3RlbWF0aWMgZWZmZWN0cyBiZXR3ZWVuIHRpbWUgcG9pbnRzCgojIyMgQW5vdmEgdGltZSBlZmZlY3RzCgpUaGUgdHlwZSBvZiBJQ0Mgd2UgaGF2ZSBjaG9zZW4gZG9lcyBub3QgdGFrZSB3aXRoaW4gc3ViamVjdCAoYmV0d2VlbiBzZXNzaW9uL3N5c3RlbWF0aWMpIHZhcmlhbmNlIGluIHRvIGFjY291bnQuIFRoaXMgaXMgd2h5IFdlaXIgcmVjb21tZW5kcyBjaGVja2luZyB3aGV0aGVyIHRoZXJlIGlzIGEgc2lnbmlmaWNhbnQgY2hhbmdlIGJhc2VkIG9uIHRpbWUgYW5kIGV4YW1pbmluZyB0aGUgU0VNcy4gVGhlc2Ugc3lzdGVtYXRpYyBlZmZlY3RzIGNvdWxkIGJlIG1lYW5pbmdmdWwgYW5kIGltcG9ydGFudCB0byBhY2NvdW50IGZvciBmb3Igc29tZSBtZWFzdXJlcyAoZS5nLiB0YXNrIG1lYXN1cmVzIHRoYXQgc2hvdyBsZWFybmluZyBlZmZlY3RzKS4KCkhhZCB3ZSBjaG9zZW4gYW5vdGhlciBraW5kIG9mIElDQyB0YWtpbmcgdGhpcyBzb3VyY2Ugb2YgdmFyaWFuY2UgaW50byBhY2NvdW50IChlLmcuIDIsMSBvciAyLGspIHRoZXkgY291bGQgaGF2ZSBzdWdnZXN0ZWQgdGhhdCB0YXNrcyBoYXZlIGxvd2VyIHJlbGlhYmlsaXR5LgoKRG9pbmcgYSBzaW1wbGUgdC10ZXN0IG9uIHRoZSBkaWZmZXJlbmNlIHNjb3JlIGFsb25lIHdvdWxkIG5vdCBiZSBhIHZlcnkgcmlnb3Vyb3VzIHdheSBvZiB0ZXN0aW5nIHdoZXRoZXIgYW55IGNoYW5nZSBpcyBtZWFuaW5nZnVsIGJlY2F1c2UgdHdvIGRpc3RyaWJ1dGlvbnMgZnJvbSBib3RoIHRpbWUgcG9pbnRzIHdpdGggZXJyb3Igd291bGQgYmUgY29tcGFyZWQgdG8gZWFjaCBvdGhlci4gRm9ydHVuYXRlbHkgdGhlcmUgYXJlIHdheXMgdG8gdGFrZSB0aGUgZXJyb3IgZm9yIGJvdGggbWVhc3VyZW1lbnRzIGluIHRvIGFjY291bnQuIAoKVG8gY2hlY2sgd2hldGhlciBhIG1lYXN1cmUgc2hvd3Mgc3lzdGVtYXRpYyBkaWZmZXJlbmNlcyBiZXR3ZWVuIHRoZSB0d28gdGltZSBwb2ludHMgaW4gYSBtZWFuaW5nZnVsIG51bWJlciBvZiBib290c3RyYXBwZWQgc2FtcGxlcyB3ZSBjYW46IGNoZWNrIGlmIHRoZSBlZmZlY3Qgb2YgdGltZSBpcyBzaWduaWZpY2FudCBpbiBlYWNoIGJvb3RzdHJhcHBlZCBzYW1wbGUgYW5kIGZpbHRlciB2YXJpYWJsZXMgdGhhdCBoYXZlIG1vcmUgdGhhbiA1JSBvZiB0aGUgYm9vc3RyYXBwZWQgc2FtcGxlcyBzaG93aW5nIHNpZ25pZmljYW50IHRpbWUgZWZmZWN0cy4KCkFub3RoZXIgd2F5IG1pZ2h0IGJlIHRvIGNvbXB1dGUgY29uZmlkZW5jZSBpbnRlcnZhbHMgdXNpbmcgU0VNcyBhcyBkZXNjcmliZWQgaW4gdGhlIHNlY29uZCBoYWxmIG9mIFdlaXIgKDIwMDUpIGFuZCBjaGVjayB3aGF0IHBlcmNlbnQgb2YgcGFydGljaXBhbnRzIGhhdmUgc2NvcmVzIHRoYXQgZmFsbCBvdXQgb2YgdGhpcyByYW5nZS4gSSBoYXZlbid0IHB1cnN1ZWQgdGhpcyBmb3Igbm93LiAKCkhlcmUgd2UgYXNrOiBXaGljaCB2YXJpYWJsZXMgaGF2ZSBzaWduaWZpY2FudCB0aW1lIGVmZmVjdHMgaW4gbW9yZSB0aGFuIDUlIG9mIHRoZSBib290c3RyYXBwZWQgc2FtcGxlcz8KCjIzLzc0IHN1cnZleSBtZWFzdXJlcwoxMzMvMzcyIHRhc2sgbWVhc3VyZXMKCmBgYHtyfQpib290X2RmICU+JQogIHNlbGVjdChkdiwgcF90aW1lLCB0YXNrKSAlPiUKICBtdXRhdGUodGltZV9lZmZlY3Rfc2lnID0gaWZlbHNlKHBfdGltZTwwLjA1LDEsMCkpICU+JQogIGdyb3VwX2J5KGR2KSU+JQogIHN1bW1hcmlzZShwY3Rfc2lnX3RpbWVfZWZmZWN0ID0gc3VtKHRpbWVfZWZmZWN0X3NpZykvMTAsCiAgICAgICAgICAgIHRhc2sgPSB1bmlxdWUodGFzaykpJT4lCiAgZmlsdGVyKHBjdF9zaWdfdGltZV9lZmZlY3Q+NSkgJT4lCiAgYXJyYW5nZSh0YXNrLC1wY3Rfc2lnX3RpbWVfZWZmZWN0KSAlPiUKICB1bmdyb3VwKCklPiUKICBncm91cF9ieSh0YXNrKSAlPiUKICBzdW1tYXJpc2UoY291bnQ9bigpKQpgYGAKCkFyZSB0aGVzZSBzaWduaWZpY2FudGx5IGRpZmZlcmVudCBiZXR3ZWVuIHN1cnZleXMgYW5kIHRhc2tzPyBOby4KCmBgYHtyfQpjaGlzcS50ZXN0KG1hdHJpeChkYXRhID0gYygyMyw3NC0yMywxMzMsIDM3Mi0xMzMpLCBucm93PTIpKQpgYGAKCkFyZSB0aGV5IHByZWRvbWluYW50bHkgaW4gb25lIG9yIHRoZSBvdGhlciBkaXJlY3Rpb24gYW5kIGRvZXMgdGhhdCBkaWZmZXIgZGVwZW5kaW5nIG9uIHN1cnZleSB2cyB0YXNrPwoKYGBge3J9CmJvb3RfZGYgJT4lCiAgc2VsZWN0KGR2LCBwX3RpbWUsIHRhc2ssIHBlYXJzb24pICU+JQogIG11dGF0ZSh0aW1lX2VmZmVjdF9zaWcgPSBpZmVsc2UocF90aW1lPDAuMDUsMSwwKSkgJT4lCiAgZ3JvdXBfYnkoZHYpJT4lCiAgc3VtbWFyaXNlKHBjdF9zaWdfdGltZV9lZmZlY3QgPSBzdW0odGltZV9lZmZlY3Rfc2lnKS8xMCwKICAgICAgICAgICAgdGFzayA9IHVuaXF1ZSh0YXNrKSwKICAgICAgICAgICAgbWVhbl9wZWFyc29uID0gbWVhbihwZWFyc29uKSklPiUKICBmaWx0ZXIocGN0X3NpZ190aW1lX2VmZmVjdD41KSAlPiUKICBhcnJhbmdlKHRhc2ssLXBjdF9zaWdfdGltZV9lZmZlY3QpICU+JQogIGFycmFuZ2UobWVhbl9wZWFyc29uKSAKICAKYGBgCgojIyMgU0VNIENJIGNhbGN1bGF0aW9uCgpTdGVwIDE6IFQgPSBHcmFuZCBtZWFuICsgSUNDICogKFN1YmplY3Qgc2NvcmUgLSBHcmFuZCBtZWFuKQpTdGVwIDI6IFNFUCA9IFNEIG9mIGJvdGggbWVhc3VyZW1lbnRzICogc3FydCgxLUlDQ14yKQoKSGVyZSBJIGNhbGN1bGF0ZSB0aGUgcHJvcG9ydGlvbiBvZiBzdWJqZWN0cyB0aGF0IG1vdmUgbW9yZSB0aGFuIG9uZSBzdGFuZGFyZCBlcnJvciBvZiBwcmVkaWN0aW9uIChTRVApIGluIHQyIGNvbXBhcmVkIHRvIHQxIGZvciBlYWNoIG1lYXN1cmUuIAoKVGhpcyBpcyBvZGQgdG8gdGhpbmsgYWJvdXQgYmVjYXVzZSB0aGUgbGFyZ2VyIHRoZSBJQ0Mgb2YgYSBtZWFzdXJlIHRoZSBzbWFsbGVyIHRoZSBTRVAuIFNvIHZlcnkgc21hbGwgZGlmZmVyZW5jZXMgYmV0d2VlbiB0aGUgdHdvIHRpbWUgcG9pbnRzIGNhbiBiZSBjYXRlZ29yaXplZCBhcyAnbWVhbmluZ2Z1bCcgYmFzZWQgb24gdGhlIHRpbnkgU0VQLgoKV2hhdCB5b3UgYXJlIGludGVyZXN0ZWQgaW4gaXMgbm90IG5lY2Vzc2FyaWx5IHdoZXRoZXIgaW5kaXZpZHVhbHMgY2hhbmdlIGF0IGFsbCBiZXR3ZWVuIHRoZSB0d28gdGltZSBwb2ludHMgdGhvdWdoLiBZb3Ugd2FudCB0byBrbm93IGlmIHRoaXMgY2hhbmdlIGlzIHN5c3RlbWF0aWMgaW4gb25lIGRpcmVjdGlvbi4KCkhlcmUgSSBjYWxjdWxhdGUgdGhlIHByb3BvcnRpb24gb2YgcGVvcGxlIHNob3dpbmcgJ21lYW5pbmdmdWwnIGNoYW5nZXMgaW4gb25lIGRpcmVjdGlvbiBvciB0aGUgb3RoZXIuIFRvIGludGVncmF0ZSBib3RoIG9mIHRoZXNlIGRpcmVjdGlvbiBJIHN1YnRyYWN0ZWQgb25lIGZyb20gdGhlIG90aGVyIGFuZCBmaWx0ZXJlZCB0aGUgdmFyaWFibGVzIHRoYXQgaGF2ZSBtb3JlIHRoYW4gNSUgb2YgdGhlIHBhcnRpY2lwYW50cyBzaG93aW5nIG1lYW5pbmdmdWwgY2hhbmdlIGluIG9uZSBkaXJlY3Rpb24gb3ZlciB0aGUgb3RoZXIgKHNvIGlmIGEgdmFyaWFibGUgaGFzIDEwIHBhcnRpY2lwYW50cyBzaG93aW5nIGRpZmZlcmVuY2UgaW4gb25lIGFuZCAxMCBpbiB0aGUgb3RoZXIgZGlyZWN0aW9uIHRoaXMgd291bGQgY2FuY2VsIG91dCBidXQgYSB2YXJpYWJsZXMgd2l0aCAxNSBwZW9wbGUgc2hvd2luZyBhIHBvc2l0aXZlIGFuZCA1IG5lZ2F0aXZlIGNoYW5nZSB3b3VsZCByZW1haW4pLgoKCmBgYHtyfQpnZXRfaW5kX2NpID0gZnVuY3Rpb24oZHZfdmFyKXsKICBtYXRjaGVkID0gbWF0Y2hfdDFfdDIoZHZfdmFyKQogIGdyYW5kX21lYW4gPSBtZWFuKG1hdGNoZWQkc2NvcmUpCiAgZ3JhbmRfc2QgPSBzZChtYXRjaGVkJHNjb3JlKQogIGR2X2ljYyA9IGdldF9pY2MoZHZfdmFyKQogIHNlcCA9IGdyYW5kX3NkICogc3FydCgxLShkdl9pY2NeMikpCiAgbWF0Y2hlZCA9IG1hdGNoZWQgJT4lIAogICAgc3ByZWFkKHRpbWUsIHNjb3JlKSAlPiUKICAgIHJlbmFtZSgidDEiPSIxIiwgInQyIj0iMiIpICU+JQogICAgbXV0YXRlKHRydWVfc2NvcmUgPSBncmFuZF9tZWFuKyhkdl9pY2MqKHQxLWdyYW5kX21lYW4pKSwKICAgICAgICAgICBjaV91cCA9IHRydWVfc2NvcmUrKDEuOTYqc2VwKSwKICAgICAgICAgICBjaV9sb3cgPSB0cnVlX3Njb3JlLSgxLjk2KnNlcCksCiAgICAgICAgICAgdDJfYWJvdmVfY2kgPSBpZmVsc2UodDI+Y2lfdXAsMSwwKSwKICAgICAgICAgICB0Ml9iZWxvd19jaSA9IGlmZWxzZSh0MjxjaV9sb3csMSwgMCkpCiAgICByZXR1cm4obWF0Y2hlZCkKfQoKZ2V0X3Byb3Bfb3V0X2NpID0gZnVuY3Rpb24oZHZfdmFyKXsKICBnZXRfaW5kX2NpKGR2X3ZhcikgJT4lCiAgICBzdW1tYXJpc2UocHJvcF9hYm92ZV9jaSA9IHN1bSh0Ml9hYm92ZV9jaSkvbigpLAogICAgICAgICAgICAgIHByb3BfYmVsb3dfY2kgPSBzdW0odDJfYmVsb3dfY2kvbigpKSkKfQoKI0NyZWF0ZSBkZiBvZiBwb2ludCBlc3RpbWF0ZSByZWxpYWJpbGl0aWVzCmluZF9jaV9kZiA8LSBkYXRhLmZyYW1lKHByb3BfYWJvdmVfY2kgPSByZXAoTkEsIGxlbmd0aChudW1lcmljX2NvbHMpKSwKICAgICAgICAgICAgICAgICAgICAgICAgcHJvcF9iZWxvd19jaSA9IHJlcChOQSwgbGVuZ3RoKG51bWVyaWNfY29scykpKQoKcm93Lm5hbWVzKGluZF9jaV9kZikgPC0gbnVtZXJpY19jb2xzCgpmb3IoaSBpbiAxOmxlbmd0aChudW1lcmljX2NvbHMpKXsKICAgIGluZF9jaV9kZltudW1lcmljX2NvbHNbaV0sICdwcm9wX2Fib3ZlX2NpJ10gPC0gZ2V0X3Byb3Bfb3V0X2NpKG51bWVyaWNfY29sc1tpXSkkcHJvcF9hYm92ZV9jaSAKICAgIGluZF9jaV9kZltudW1lcmljX2NvbHNbaV0sICdwcm9wX2JlbG93X2NpJ10gPC0gZ2V0X3Byb3Bfb3V0X2NpKG51bWVyaWNfY29sc1tpXSkkcHJvcF9iZWxvd19jaSAKfQoKaW5kX2NpX2RmJGR2ID0gcm93Lm5hbWVzKGluZF9jaV9kZikKcm93Lm5hbWVzKGluZF9jaV9kZikgPSBzZXEoMTpucm93KGluZF9jaV9kZikpCmluZF9jaV9kZiR0YXNrID0gJ3Rhc2snCmluZF9jaV9kZltncmVwKCdzdXJ2ZXknLCBpbmRfY2lfZGYkZHYpLCAndGFzayddID0gJ3N1cnZleScKaW5kX2NpX2RmW2dyZXAoJ2hvbHQnLCBpbmRfY2lfZGYkZHYpLCAndGFzayddID0gInRhc2siCmluZF9jaV9kZiA9IGluZF9jaV9kZiAlPiUKICBzZWxlY3QoZHYsIHRhc2ssIHByb3BfYWJvdmVfY2ksIHByb3BfYmVsb3dfY2kpCgptZWFuX2RpZmZfZGYgPSBpbmRfY2lfZGYgJT4lCiAgbXV0YXRlKHByb3Bfb25lX2RpcmVjdGlvbiA9IHByb3BfYWJvdmVfY2ktcHJvcF9iZWxvd19jaSkgJT4lCiAgZmlsdGVyKGFicyhwcm9wX29uZV9kaXJlY3Rpb24pPjAuMDUpICU+JQogIGFycmFuZ2UoLWFicyhwcm9wX29uZV9kaXJlY3Rpb24pKQoKbWVhbl9kaWZmX2RmCgojIHdyaXRlLmNzdihtZWFuX2RpZmZfZGYsICcvVXNlcnMvemV5bmVwZW5rYXZpL0Ryb3Bib3gvUG9sZHJhY2tMYWIvU1JPX1JldGVzdF9BbmFseXNlcy9vdXRwdXQvdGFibGVzL21lYW5fZGlmZl9kZi5jc3YnKQpgYGAKClRoaXMgZG9lc24ndCBtYWtlIGl0IGVhc2llciB0byBpbnRlcnByZXQgd2hldGhlciBpdCBpcyBhIHBlcmZvcm1hbmNlIGltcHJvdmVtZW50IG9yIGRlY2xpbmUuIEluIGZhY3QgJ3BlcmZvcm1hbmNlJyBpc24ndCBldmVuIG5lY2Vzc2FyaWx5IHRoZSBjb3JyZWN0IHRlcm0gaGVyZS4gVXNpbmcgdGhlIHZhbGVuY2UgYXNzaWdubWVudHMgb2YgdGhlIG1lYXN1cmVzIGxvb2sgYXQgd2hldGhlciBpdCB0cmFuc2xhdGVzIHRvIGFuIGluY3JlYXNlIG9yIGRlY3JlYXNlIGluIHNlbGYtcmVndWxhdGlvbi4gT2YgdGhlIDY2IHRoaXMgbWFrZXMgc2Vuc2UgZm9yIDU0IHZhcmlhYmxlcy4KCmBgYHtyIHdhcm5pbmc9RkFMU0UsIG1lc3NhZ2U9RkFMU0V9Cm1lYW5fZGlmZl9kZiA9IHJlYWQuY3N2KCcvVXNlcnMvemV5bmVwZW5rYXZpL0Ryb3Bib3gvUG9sZHJhY2tMYWIvU1JPX1JldGVzdF9BbmFseXNlcy9vdXRwdXQvdGFibGVzL21lYW5fZGlmZl9kZi5jc3YnKQoKdGFibGUoYXMuY2hhcmFjdGVyKG1lYW5fZGlmZl9kZiR2YWxlbmNlKSwgdXNlTkEgPSAiaWZhbnkiKQoKbWVhbl9kaWZmX2RmICU+JQogIG11dGF0ZShzY19pbmNyZWFzZSA9IGlmZWxzZShwcm9wX29uZV9kaXJlY3Rpb24+MCZ2YWxlbmNlPT0iUG9zIiwxLGlmZWxzZShwcm9wX29uZV9kaXJlY3Rpb248MCZ2YWxlbmNlPT0iTmVnIiwxLDApKSwKICAgICAgICAgc2NfZGVjcmVhc2UgPSBpZmVsc2UocHJvcF9vbmVfZGlyZWN0aW9uPjAmdmFsZW5jZT09Ik5lZyIsMSxpZmVsc2UocHJvcF9vbmVfZGlyZWN0aW9uPDAmdmFsZW5jZT09IlBvcyIsMSwwKSkpICU+JQpzdW1tYXJpc2Uoc3VtX3NjX2luY3JlYXNlID0gc3VtKHNjX2luY3JlYXNlLG5hLnJtPVQpLAogICAgICAgICAgc3VtX3NjX2RlY3JlYXNlID0gc3VtKHNjX2RlY3JlYXNlLG5hLnJtPVQpKQoKYGBgCgpWYXJpYWJsZXMgdGhhdCBzaG93IG1vcmUgdGhhbiA1JSBkaWZmZXJlbmNlIGluIGEgZGlyZWN0aW9uIGRvIG5vdCBkZXBlbmQgb24gd2hldGhlciB0aGV5IHRyYW5zbGF0ZSB0byBhIHNlbGYgY29udHJvbCBpbmNyZWFzZSBvciBkZWNyZWFzZS4KCmBgYHtyfQpjaGlzcS50ZXN0KG1hdHJpeChkYXRhID0gYygyMyw1NC0yMywzMSw1NC0zMSksIG5yb3c9MikpCmBgYAoKRG8gdGhleSBkaWZmZXIgZGVwZW5kaW5nIG9uIHdoZXRoZXIgdGhleSBhcmUgdGFzayBvciBzdXJ2ZXkgdmFyaWFibGVzPyBOby4KCmBgYHtyfQpjaGlzcS50ZXN0KG1hdHJpeChkYXRhID0gYyg1LDc0LTUsNjEsMzcyLTYxKSwgbnJvdz0yKSkKYGBgCgpgYGB7cn0Kcm0obWVhbl9kaWZmX2RmKQpgYGAKCiMjIFRhc2sgUmVsaWFiaWxpdGllcwoKSGVyZSB3ZSBzdW1tYXJpemUgdGhlIHJlc3VsdHMgb24gYSB0YXNrIGxldmVsIHRvIG1ha2UgaXQgbW9yZSBkaWdlc3RhYmxlIGFuZCBlYXNpZXIgdG8gbWFrZSBjb250YWN0IHdpdGggdGhlIGxpdGVyYXR1cmUuICAKCldlIHJlZHVjZSB0aGUgbGlzdCBvZiB0YXNrIG1lYXN1cmVzIHRvIGEgbGlzdCBvZiBvbmUgcGVyIHRhc2sgYnkgYXZlcmFnaW5nIG9ubHkgdGhlIHJhdyBtZWFzdXJlcyBmcm9tIGFsbCB0aGUgdHJpYWxzIGluIGEgdGFzay4gV2UgY2hvc2UgdG8gcmVkdWNlIHRoZSBpbmZvcm1hdGlvbiBpbiB0aGlzIG1hbm5lciB0byBhdm9pZCBhbnkgYmlhcyBzdGVtbWluZyBmcm9tIGRpZmZlcmVudGlhbCBhbW91bnQgb2YgaW50ZXJlc3QgYW5kIHByb2NlZHVyZXMgYXBwbGllZCB0byBjZXJ0YWluIHRhc2tzIG92ZXIgb3RoZXJzIChlLmcuIGEgdGFzayBjYW4gaGF2ZSBvdmVyIDEwIG1lYXN1cmVzIGJlY2F1c2UgaXQgaGFzIG11bHRpcGxlIGNvbmRpdGlvbnMgYW5kIHdlIGhhdmUgY2hvc2VuIHRvIGZpdCBERE0ncyBmb3Igc3BlY2lmaWMgY29uZGl0aW9ucyB3aGlsZSBhbm90aGVyIG1pZ2h0IG9ubHkgaGF2ZSAyIGR1ZSB0byBvdXIgcmVsYXRpdmUgaW5leHBlcmllbmNlIGFuZCBsYWNrIG9mIGludGVyZXN0IGluIGl0KS4gV2UgY2hlY2sgd2hldGhlciB0aGUgbnVtYmVyIG9mIHRyaWFscyBpbiBhIHRhc2sgaGFzIGEgc2lnbmlmaWNhbnQgZWZmZWN0IG9uIHRoZXNlIGF2ZXJhZ2UgcmVsaWFiaWxpdGllcyBvZiByYXcgbWVhc3VyZXMgYXMgd2VsbC4gCgpXZSBmaWx0ZXIgb3V0IHRoZSBERE0gcGFyYW1ldGVycyBhbmQgbWVhc3VyZXMgZm9yIHNwZWNpZmljIGNvbnRyYXN0cy4gTm90ZSB0aGF0IHRoaXMgZG9lcyBsZWF2ZSBzb21lIHRhc2tzIHdpdGggbWVhc3VyZXMgdGhhdCBhcmUgbW9kZWwgZml0cyBhbmQvb3IgZm9yIHNwZWNpZmljIGNvbmRpdGlvbnMgKGJlY2F1c2UgYXQgbGVhc3QgdGhlIGN1cnJlbnQgZGF0YXNldHMgZG8gbm90IGluY2x1ZGUgbWVhc3VyZXMgdGhhdCBhcmUgYmFzZWQgb24gYWxsIHRoZSB0cmlhbHMgKiphbmQqKiBhcmUgcmF3IHRob3VnaCBJIGNvdWxkIGRpdmUgaW4gdG8gdmFyaWFibGVzX2V4aGF1c3RpdmUgZm9yIHN1Y2ggbWVhc3VyZXMuIEZvciBleGFtcGxlIHRoZSBhdmVyYWdlIHJlbGlhbGliaWxpdHkgZm9yIEtpcmJ5IGlzIGJhc2VkIG9uIHRocmVlIGRpc2NvdW50IHJhdGVzIGZvciBzcGVjaWZpYyBjb25kaXRpb25zLikuIEhlcmUncyB0aGUgb3JkZXIgb2YgdGFza3MgYnkgbWVhbiByZWxpYWJpbGl0eSBzb3J0ZWQgZm9yIElDQyBhbmQgdGhlbiBTcGVhcm1hbidzICRccmhvJC4KCmBgYHtyIHdhcm5pbmc9RkFMU0UsIG1lc3NhZ2U9RkFMU0V9CnRtcCA9IG1lYXN1cmVfbGFiZWxzICU+JQogIG11dGF0ZShkdiA9IGFzLmNoYXJhY3RlcihkdikpICU+JQogIGZpbHRlcih0YXNrID09ICd0YXNrJykgJT4lCiAgbGVmdF9qb2luKGJvb3RfZGZbLGMoImR2IiwgImljYyIsICJzcGVhcm1hbiIpXSwgYnkgPSAnZHYnKSAlPiUKICBmaWx0ZXIob3ZlcmFsbF9kaWZmZXJlbmNlICE9ICdkaWZmZXJlbmNlJyAmIHJhd19maXQgJWluJSBjKCdFWicsICdoZGRtJykgPT0gRkFMU0UpJT4lCiAgc2VwYXJhdGUoZHYsIGMoJ3Rhc2tfbmFtZScsICdleHRyYV8xJywgJ2V4dHJhXzInKSwgc2VwID0gJ1xcLicscmVtb3ZlPUZBTFNFKSAlPiUKICBzZWxlY3QoLWV4dHJhXzEsIC1leHRyYV8yKSAlPiUKICBncm91cF9ieSh0YXNrX25hbWUpICU+JQogIHN1bW1hcmlzZShtZWRpYW5faWNjID0gbWVkaWFuKGljYyksCiAgICAgICAgICAgIG1lZGlhbl9zcGVhcm1hbiA9IG1lZGlhbihzcGVhcm1hbiksCiAgICAgICAgICAgIGljY18yLjUgPSBxdWFudGlsZShpY2MsIHByb2JzID0gMC4wMjUpLAogICAgICAgICAgICBpY2NfOTcuNSA9IHF1YW50aWxlKGljYywgcHJvYnMgPSAwLjk3NSksCiAgICAgICAgICAgIG1pbl9zcGVhcm1hbiA9IG1pbihzcGVhcm1hbiksCiAgICAgICAgICAgIG1heF9zcGVhcm1hbiA9IG1heChzcGVhcm1hbiksCiAgICAgICAgICAgIG51bV9tZWFzdXJlcyA9IG4oKS8xMDAwLAogICAgICAgICAgICBtZWFuX251bV90cmlhbHMgPSByb3VuZChtZWFuKG51bV9hbGxfdHJpYWxzKSkpJT4lCiAgYXJyYW5nZSgtbWVkaWFuX2ljYywgLW1lZGlhbl9zcGVhcm1hbikKCnRtcCAlPiUKICBkYXRhdGFibGUoKSAlPiUKICBmb3JtYXRSb3VuZChjb2x1bW5zPWMoJ21lZGlhbl9zcGVhcm1hbicsICdtZWRpYW5faWNjJywKICAgICAgICAgICAgICAgICAgICAgICAgJ21pbl9zcGVhcm1hbicsICdpY2NfMi41JywKICAgICAgICAgICAgICAgICAgICAgICAgJ21heF9zcGVhcm1hbicsICdpY2NfOTcuNScpLCBkaWdpdHM9MykKYGBgCgoKYGBge3J9CnRtcCA9IHRtcCU+JQogIHNlbGVjdCgtbWVkaWFuX3NwZWFybWFuLCAtbWluX3NwZWFybWFuLCAtbWF4X3NwZWFybWFuKSAlPiUKICBtdXRhdGUodGFza19uYW1lID0gZ3N1YigiXyIsICIgIiwgdGFza19uYW1lKSwKICAgICAgICAgdGFza19uYW1lID0gZ3N1YigiKF58W1s6c3BhY2U6XV0pKFtbOmFscGhhOl1dKSIsICJcXDFcXFVcXDIiLCB0YXNrX25hbWUsIHBlcmw9VFJVRSkpCgpuYW1lcyh0bXApID0gZ3N1YigiXyIsICIgIiwgbmFtZXModG1wKSkKbmFtZXModG1wKSA9IGdzdWIoIihefFtbOnNwYWNlOl1dKShbWzphbHBoYTpdXSkiLCAiXFwxXFxVXFwyIiwgbmFtZXModG1wKSwgcGVybD1UUlVFKQoKc2p0LmRmKHRtcCAlPiUgbXV0YXRlX2lmKGlzLm51bWVyaWMsIGZ1bnMocm91bmQoLiwgMykpKSwgZGVzY3JpYmU9RiwgaGlkZS5wcm9ncmVzcyA9IFRSVUUsIHNob3cucm93bmFtZXMgPSBGQUxTRSwgZmlsZSA9ICIvVXNlcnMvemV5bmVwZW5rYXZpL0Ryb3Bib3gvUG9sZHJhY2tMYWIvU1JPX1JldGVzdF9BbmFseXNlcy9vdXRwdXQvdGFibGVzL3Rhc2tfcmVsX3RhYmxlLmRvYyIpCmBgYAoKIyMjIE51bWJlciBvZiB0cmlhbHMKCkRvZXMgbnVtYmVyIG9mIGl0ZW1zIGluIGEgdGFzayBoYXZlIGEgc2lnbmlmaWNhbnQgZWZmZWN0IG9uIHRoZSBhdmVyYWdlIElDQyBvZiAobW9zdGx5KSByYXcgbWVhc3VyZXMgZm9yIGFsbCB0cmlhbHMgZnJvbSBhIHRhc2s/IE5vLiAobm8gZWZmZWN0IG9uIFNwZWFybWFuIGVpdGhlcikKCmBgYHtyIHdhcm5pbmc9RkFMU0UsIG1lc3NhZ2U9RkFMU0V9CnRtcCA9IG1lYXN1cmVfbGFiZWxzICU+JQogIG11dGF0ZShkdiA9IGFzLmNoYXJhY3RlcihkdikpICU+JQogIGZpbHRlcih0YXNrID09ICd0YXNrJykgJT4lCiAgIyBsZWZ0X2pvaW4ocmVsX2RmWyxjKCJkdiIsICJzcGVhcm1hbiIsImljYyIpXSwgYnk9J2R2JykgJT4lCiAgbGVmdF9qb2luKGJvb3RfZGZbLGMoImR2IiwgInNwZWFybWFuIiwiaWNjIildLCBieT0nZHYnKSAlPiUKICBmaWx0ZXIob3ZlcmFsbF9kaWZmZXJlbmNlICE9ICdkaWZmZXJlbmNlJyAmIHJhd19maXQgJWluJSBjKCdFWicsICdoZGRtJykgPT0gRkFMU0UpJT4lCiAgc2VwYXJhdGUoZHYsIGMoJ3Rhc2tfbmFtZScsICdleHRyYV8xJywgJ2V4dHJhXzInKSwgc2VwID0gJ1xcLicscmVtb3ZlPUZBTFNFKSAlPiUKICBzZWxlY3QoLWV4dHJhXzEsIC1leHRyYV8yKQoKIyBzdW1tYXJ5KGxtKGljYyB+IG51bV9hbGxfdHJpYWxzLCBkYXRhID0gdG1wKSkKc3VtbWFyeShsbWVyVGVzdDo6bG1lcihpY2MgfiBudW1fYWxsX3RyaWFscyArICgxfGR2KSwgZGF0YSA9IHRtcCkpCmBgYAoKYGBge3Igd2FybmluZz1GQUxTRSwgbWVzc2FnZT1GQUxTRX0KbWVhc3VyZV9sYWJlbHMgJT4lCiAgbXV0YXRlKGR2ID0gYXMuY2hhcmFjdGVyKGR2KSkgJT4lCiAgZmlsdGVyKHRhc2sgPT0gJ3Rhc2snKSAlPiUKICAjIGxlZnRfam9pbihyZWxfZGZbLGMoImR2IiwgInNwZWFybWFuIiwiaWNjIildLCBieT0nZHYnKSAlPiUKICBsZWZ0X2pvaW4oYm9vdF9kZlssYygiZHYiLCAic3BlYXJtYW4iLCJpY2MiKV0sIGJ5PSdkdicpICU+JQogIGZpbHRlcihvdmVyYWxsX2RpZmZlcmVuY2UgIT0gJ2RpZmZlcmVuY2UnICYgcmF3X2ZpdCAlaW4lIGMoJ0VaJywgJ2hkZG0nKSA9PSBGQUxTRSklPiUKICBzZXBhcmF0ZShkdiwgYygndGFza19uYW1lJywgJ2V4dHJhXzEnLCAnZXh0cmFfMicpLCBzZXAgPSAnXFwuJyxyZW1vdmU9RkFMU0UpICU+JQogIHNlbGVjdCgtZXh0cmFfMSwgLWV4dHJhXzIpICU+JQogIGdncGxvdChhZXMobnVtX2FsbF90cmlhbHMsIGljYykpKwogIGdlb21fcG9pbnQoKSsKICBnZW9tX3Ntb290aChtZXRob2Q9ImxtIikrCiAgdGhlbWVfYncoKSsKICB4bGFiKCJOdW1iZXIgb2YgdHJpYWxzIikrCiAgeWxhYigiSUNDIikKYGBgCgojIyMjIFRyaWFsIG51bWJlciBkZXBlbmRlbmNlIGludHJhbWVhc3VyZSAKClRoZSBhYm92ZSBhbmFseXNpcyB3YXMgbG9va2luZyBhdCB0aGUgZWZmZWN0IG9mIG51bWJlciBvZiB0cmlhbHMgYWNyb3NzIHRhc2tzLiBCdXQgc29tZSB0YXNrcyBtaWdodCBiZSBiYWQgZm9yIGluZGl2aWR1YWwgZGlmZmVyZW5jZSBtZWFzdXJlbWVudCByZWdhcmRsZXNzIG9mIGhvdyBtYW55IHRyaWFscyB0aGVyZSBhcmUgaW4gdGhlbSB3aGVyZWFzIGZvciBvdGhlcnMgZmV3ZXIgdHJpYWxzIG1pZ2h0IGJlIHlpZWxkaW5nIGEgc3VmZmljaWVudGx5IHJlbGlhYmxlIG1lYXN1cmUuCgpGb3IgdGFza3MgZm9yIHdoaWNoIGRlcGVuZGVudCB2YXJpYWJsZXMgYXJlIGVzdGltYXRlZCB1c2luZyBtYW55IHRyaWFscyBvbmUgY2FuIGFzazogRG9lcyB0aGUgc2FtZSBtZWFzdXJlIGdldCBsZXNzIHJlbGlhYmxlIGlmIGZld2VyIHRyaWFscyBhcmUgdXNlZCB0byBlc3RpbWF0ZSBpdHMgcmVsaWFiaWxpdHk/CgpUaGlzIHdvbid0IG1ha2Ugc2Vuc2UgZm9yIGFsbCB0YXNrcy4gRm9yIGV4YW1wbGUgdG8gZXN0aW1hdGUgYSByaXNrIGF2ZXJzaW9uIHBhcmFtZXRlciB5b3UgbmVlZCBhbGwgdHJpYWxzIGZvciBIb2x0IGFuZCBMYXVyeS4gRm9yIEtpcmJ5IGFuZCBCaWNrZWwgeW91IGhhdmUgc3BlY2lmaWMgY29uZGl0aW9ucyBsb29raW5nIGF0IGZld2VyIHRyaWFscy4gVGhlIENvZ25pdGl2ZSBSZWZsZWN0aW9uIFRhc2sgbWlnaHQgYmUgbW9yZSBhcHByb3ByaWF0ZSB0byBhbmFseXplIGVhY2ggaXRlbSBzZWFwcmF0ZWx5LiBUaGUgd3JpdGluZyB0YXNrIGRvZXMgbm90IGhhdmUgdHJpYWwgbnVtYmVycy4gRm9yIGFsbCBvdGhlcnMgaXQgbWlnaHQgYmUgaW50ZXJlc3RpbmcgdG8gaW52ZXN0aWdhdGUuCgpUaGVzZSBraW5kcyBvZiBhbmFseXNlcyBhcmUgdG9vIHRhc2stc3BlY2lmaWMgYW5kIGluLWRlcHRoIGZvciBhIHBhcGVyIHRoYXQgaXMgdHJ5aW5nIHRvIGdpdmUgYSBnbG9iYWwgc2Vuc2Ugb2YgdGhlIGRpZmZlcmVuY2VzIGJldHdlZW4gc2VsZi1yZWd1bGF0aW9uIG1lYXN1cmVzIGluIHRoZWlyIHN1aXRhYmxpdHkgZm9yIGluZGl2aWR1YWwgZGlmZmVyZW5jZSBhbmFseXNlcyBiYXNlZCBvbiB0aGVpciBzdGFiaWxpdHkgYWNyb3NzIHRpbWUuIFN1Y2ggYW5hbHlzZXMgd291bGQgcHJvdmlkZSBhIGRldGFpbGVkIGV4YW1pbmF0aW9uIG9mIGhvdyB0byBleHRyYWN0IHRoZSBtb3N0IHJlbGlhYmxlL2Jlc3QgaW5kaXZpZHVhbCBkaWZmZXJlbmNlIG1lYXN1cmUgZnJvbSB0YXNrcyB3aXRoIGEgc2V0IG9mIG1lZGlvY3JlIHZhcmlhYmxlcyB0byBiZWdpbiB3aXRoLiBUaG91Z2ggd2UgZG8gbm90IHByb3ZpZGUgc3VjaCBhIGNvbXByZWhlbnNpdmUgYW5hbHlzaXMgb2YgdGhpcyBzb3J0IGluIHRoaXMgcGFwZXIgd2UgcHJvdmlkZSBhIHNpbmdsZSBleGFtcGxlIG9mIHRoaXMgYXBwcm9hY2ggYW5kIGhvcGUgdGhlIG9wZW4gYWNjZXNzIHdlIHByb3ZpZGUgdG8gdGhlIGRhdGEgc3B1cnMgZnVydGhlciB3b3JrLgoKRm9yIHRoaXMgZXhhbXBsZSB3ZSBsb29rIGF0IHRoZSByZXRlc3QgcmVsaWFiaWxpdHkgb2YgdGhlIHRocmVlIGRlcGVuZGVudCBtZWFzdXJlcyAoYXZlcmFnZSBhY2N1cmFjeSwgbWVkaWFuIHJlc3BvbnNlIHRpbWUsIHRvdGFsIHNjb3JlKSBmcm9tIHRoZSBoaWVyYXJjaGljYWwgcnVsZSB0YXNrIHdpdGggMzYwIHRyaWFscy4gSGVyZSBpcyBhIGdyYXBoIG9mIGhvdyB0aGUgcG9pbnQgZXN0aW1hdGVzIG9mIHRoZSByZXRlc3QgcmVsaWFiaWxpdHkgY2hhbmdlcyBmb3IgZWFjaCBvZiB0aGUgZGVwZW5kZW50IG1lYXN1cmVzIHVzaW5nIGRpZmZlcmVudCBudW1iZXJzIG9mIHRyaWFscyB0byBlc3RpbWF0ZSB0aGVtLiBXaGlsZSB0aGUgcmVsaWFiaWxpdHkgZXN0aW1hdGUgZm9yIGVhY2ggb2YgdGhlIHZhcmlhYmxlcyByZXNwZWN0aXZlbHkgYXJlIC4uLiwgLi4uIGFuZCAuLi4gdXNpbmcgYWxsIG9mIHRoZSB0cmlhbHMsIHRoZXNlIHZhbHVlcyBhcmUgaW5kZXBlbmRlbnQgb2YgbnVtYmVyIG9mIHRyaWFscyB1c2VkIGluIHRoZSB0YXNrIGZvciBvbmx5IGNlcnRhaW4gdmFyaWFibGVzLiBCYXNlZCBvbiB0aGlzIGFuYWx5c2lzIHJlc2VhcmNoZXJzIG1pZ2h0IGRlY2lkZSB0byB1c2UgYSB2ZXJzaW9uIG9mIHRoZSB0YXNrIHdpdGggZmV3ZXIgdHJpYWxzIG9yIG9ubHkgbWVhc3VyZXMgd2l0aCBjb25zaXN0ZW50bHkgaGlnaCByZWxpYWJpbGl0eSBlc3RpbWF0ZXMuCgpFeGFtcGxlIHRhc2s6IHRocmVlIGJ5IHR3bwoKUG9zdCBwcm9jZXNzIGR2J3MgZnJvbSBjbHVzdGVyIHRvIGNhbGN1bGF0ZSByZWxpYWJpbGl0aWVzCmBgYHtyfQp0MV9kdnMgPSByZWFkLmNzdihwYXN0ZTAocmV0ZXN0X2RhdGFfcGF0aCwgJ3QxX3RidF9kdnMuY3N2JykpCnQyX2R2cyA9IHJlYWQuY3N2KHBhc3RlMChyZXRlc3RfZGF0YV9wYXRoLCAndDJfdGJ0X2R2cy5jc3YnKSkKCnQxX2R2cyA9IHQxX2R2cyAlPiUgc2VsZWN0KC1YKQp0Ml9kdnMgPSB0Ml9kdnMgJT4lIHNlbGVjdCgtWCkKYGBgCgpgYGB7cn0KaHJfbWVyZ2UgPSBtZXJnZSh0MV9kdnMsIHQyX2R2cywgYnkgPSBjKCJzdWJfaWQiLCAiYnJlYWtzIikpCgpocl9tZXJnZSA9IGhyX21lcmdlICU+JQogIGdhdGhlcihrZXksIHZhbHVlLCAtc3ViX2lkLCAtYnJlYWtzKSAlPiUKICBzZXBhcmF0ZShrZXksIGMoImR2IiwgInRpbWUiKSwgc2VwPSJcXC4iKSAlPiUKICBtdXRhdGUodGltZSA9IGlmZWxzZSh0aW1lID09ICJ4IiwgMSwgMikpCgp0MV9kdnMgPSBocl9tZXJnZSAlPiUKICBmaWx0ZXIodGltZSA9PSAxKSAlPiUKICBzZWxlY3QoLXRpbWUpICU+JQogIHNwcmVhZChkdiwgdmFsdWUpCgp0Ml9kdnMgPSBocl9tZXJnZSAlPiUKICBmaWx0ZXIodGltZSA9PSAyKSAlPiUKICBzZWxlY3QoLXRpbWUpICU+JQogIHNwcmVhZChkdiwgdmFsdWUpCgojIGNhbGN1bGF0ZSBwb2ludCBlc3RpbWF0ZXMgZm9yIHJlbGlhYmlsaXR5IG9mIGVhY2ggb2YgdGhlIHZhcmlhYmxlcyBmb3IgZWFjaCBicmVhawojIGdldF9pY2MgZm9yIGVhY2ggYnJlYWsgb2YgdG1wX3QxX2R2cyBhbmQgdG1wX3QyX2R2cwoKdHJpYWxfbnVtX3JlbF9kZiA9IGRhdGEuZnJhbWUoYnJlYWtzPXJlcChOQSwgbGVuZ3RoKHVuaXF1ZSh0MV9kdnMkYnJlYWtzKSkpLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICBhY2NfaWNjPXJlcChOQSwgbGVuZ3RoKHVuaXF1ZSh0MV9kdnMkYnJlYWtzKSkpLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICBhdmdfcnRfZXJyb3JfaWNjPXJlcChOQSwgbGVuZ3RoKHVuaXF1ZSh0MV9kdnMkYnJlYWtzKSkpLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdGRfcnRfZXJyb3JfaWNjPXJlcChOQSwgbGVuZ3RoKHVuaXF1ZSh0MV9kdnMkYnJlYWtzKSkpLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICBhdmdfcnRfaWNjPXJlcChOQSwgbGVuZ3RoKHVuaXF1ZSh0MV9kdnMkYnJlYWtzKSkpLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdGRfcnRfaWNjPXJlcChOQSwgbGVuZ3RoKHVuaXF1ZSh0MV9kdnMkYnJlYWtzKSkpLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICBtaXNzZWRfcGVyY2VudF9pY2M9cmVwKE5BLCBsZW5ndGgodW5pcXVlKHQxX2R2cyRicmVha3MpKSksCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGN1ZV9zd2l0Y2hfY29zdF9ydF8xMDBfaWNjPXJlcChOQSwgbGVuZ3RoKHVuaXF1ZSh0MV9kdnMkYnJlYWtzKSkpLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjdWVfc3dpdGNoX2Nvc3RfcnRfOTAwX2ljYz1yZXAoTkEsIGxlbmd0aCh1bmlxdWUodDFfZHZzJGJyZWFrcykpKSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdGFza19zd2l0Y2hfY29zdF9ydF8xMDBfaWNjPXJlcChOQSwgbGVuZ3RoKHVuaXF1ZSh0MV9kdnMkYnJlYWtzKSkpLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0YXNrX3N3aXRjaF9jb3N0X3J0XzkwMF9pY2M9cmVwKE5BLCBsZW5ndGgodW5pcXVlKHQxX2R2cyRicmVha3MpKSksCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGN1ZV9zd2l0Y2hfY29zdF9hY2NfMTAwX2ljYz1yZXAoTkEsIGxlbmd0aCh1bmlxdWUodDFfZHZzJGJyZWFrcykpKSwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY3VlX3N3aXRjaF9jb3N0X2FjY185MDBfaWNjPXJlcChOQSwgbGVuZ3RoKHVuaXF1ZSh0MV9kdnMkYnJlYWtzKSkpLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0YXNrX3N3aXRjaF9jb3N0X2FjY18xMDBfaWNjPXJlcChOQSwgbGVuZ3RoKHVuaXF1ZSh0MV9kdnMkYnJlYWtzKSkpLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0YXNrX3N3aXRjaF9jb3N0X2FjY185MDBfaWNjPXJlcChOQSwgbGVuZ3RoKHVuaXF1ZSh0MV9kdnMkYnJlYWtzKSkpKQoKZm9yKGkgaW4gMTpsZW5ndGgodW5pcXVlKHQxX2R2cyRicmVha3MpKSl7CiAgY3VyX2JyZWFrID0gdW5pcXVlKHQxX2R2cyRicmVha3MpW2ldCiAgdG1wX3QxX2R2cyA9IHQxX2R2cyAlPiUgZmlsdGVyKGJyZWFrcyA9PSBjdXJfYnJlYWspCiAgdG1wX3QyX2R2cyA9IHQyX2R2cyAlPiUgZmlsdGVyKGJyZWFrcyA9PSBjdXJfYnJlYWspCiAgdHJpYWxfbnVtX3JlbF9kZiRicmVha3NbaV0gPSBjdXJfYnJlYWsKICB0cmlhbF9udW1fcmVsX2RmJGFjY19pY2NbaV0gPSBnZXRfaWNjKCJhY2MiLCB0bXBfdDFfZHZzLCB0bXBfdDJfZHZzKQogIHRyaWFsX251bV9yZWxfZGYkYXZnX3J0X2Vycm9yX2ljY1tpXSA9IGdldF9pY2MoImF2Z19ydF9lcnJvciIsIHRtcF90MV9kdnMsIHRtcF90Ml9kdnMpCiAgdHJpYWxfbnVtX3JlbF9kZiRzdGRfcnRfZXJyb3JfaWNjW2ldID0gZ2V0X2ljYygic3RkX3J0X2Vycm9yIiwgdG1wX3QxX2R2cywgdG1wX3QyX2R2cykKICB0cmlhbF9udW1fcmVsX2RmJGF2Z19ydF9pY2NbaV0gPSBnZXRfaWNjKCJhdmdfcnQiLCB0bXBfdDFfZHZzLCB0bXBfdDJfZHZzKQogIHRyaWFsX251bV9yZWxfZGYkc3RkX3J0X2ljY1tpXSA9IGdldF9pY2MoInN0ZF9ydCIsIHRtcF90MV9kdnMsIHRtcF90Ml9kdnMpCiAgdHJpYWxfbnVtX3JlbF9kZiRtaXNzZWRfcGVyY2VudF9pY2NbaV0gPSBnZXRfaWNjKCJtaXNzZWRfcGVyY2VudCIsIHRtcF90MV9kdnMsIHRtcF90Ml9kdnMpCiAgdHJpYWxfbnVtX3JlbF9kZiRjdWVfc3dpdGNoX2Nvc3RfcnRfMTAwX2ljY1tpXSA9IGdldF9pY2MoImN1ZV9zd2l0Y2hfY29zdF9ydF8xMDAiLCB0bXBfdDFfZHZzLCB0bXBfdDJfZHZzKQogIHRyaWFsX251bV9yZWxfZGYkY3VlX3N3aXRjaF9jb3N0X3J0XzkwMF9pY2NbaV0gPSBnZXRfaWNjKCJjdWVfc3dpdGNoX2Nvc3RfcnRfOTAwIiwgdG1wX3QxX2R2cywgdG1wX3QyX2R2cykKICB0cmlhbF9udW1fcmVsX2RmJHRyaWFsX3N3aXRjaF9jb3N0X3J0XzEwMF9pY2NbaV0gPSBnZXRfaWNjKCJ0YXNrX3N3aXRjaF9jb3N0X3J0XzEwMCIsIHRtcF90MV9kdnMsIHRtcF90Ml9kdnMpCiAgdHJpYWxfbnVtX3JlbF9kZiR0cmlhbF9zd2l0Y2hfY29zdF9ydF85MDBfaWNjW2ldID0gZ2V0X2ljYygidGFza19zd2l0Y2hfY29zdF9ydF85MDAiLCB0bXBfdDFfZHZzLCB0bXBfdDJfZHZzKQogIHRyaWFsX251bV9yZWxfZGYkY3VlX3N3aXRjaF9jb3N0X2FjY18xMDBfaWNjW2ldID0gZ2V0X2ljYygiY3VlX3N3aXRjaF9jb3N0X2FjY18xMDAiLCB0bXBfdDFfZHZzLCB0bXBfdDJfZHZzKQogIHRyaWFsX251bV9yZWxfZGYkY3VlX3N3aXRjaF9jb3N0X2FjY185MDBfaWNjW2ldID0gZ2V0X2ljYygiY3VlX3N3aXRjaF9jb3N0X2FjY185MDAiLCB0bXBfdDFfZHZzLCB0bXBfdDJfZHZzKQogIHRyaWFsX251bV9yZWxfZGYkdHJpYWxfc3dpdGNoX2Nvc3RfYWNjXzEwMF9pY2NbaV0gPSBnZXRfaWNjKCJ0YXNrX3N3aXRjaF9jb3N0X2FjY18xMDAiLCB0bXBfdDFfZHZzLCB0bXBfdDJfZHZzKQogIHRyaWFsX251bV9yZWxfZGYkdHJpYWxfc3dpdGNoX2Nvc3RfYWNjXzkwMF9pY2NbaV0gPSBnZXRfaWNjKCJ0YXNrX3N3aXRjaF9jb3N0X2FjY185MDAiLCB0bXBfdDFfZHZzLCB0bXBfdDJfZHZzKQp9CnJtKGksIGN1cl9icmVhaywgdG1wX3QxX2R2cywgdG1wX3QyX2R2cykKCnRyaWFsX251bV9yZWxfZGYkYnJlYWtzID0gYXMubnVtZXJpYyh0cmlhbF9udW1fcmVsX2RmJGJyZWFrcykKCndyaXRlLmNzdih0cmlhbF9udW1fcmVsX2RmLCBwYXN0ZTAocmV0ZXN0X2RhdGFfcGF0aCwgJ3RyaWFsX251bV9yZWxfZGZfdGJ0LmNzdicpKQpgYGAKCmBgYHtyIHdhcm5pbmc9RkFMU0UsIG1lc3NhZ2U9RkFMU0V9CiMgdHJpYWxfbnVtX3JlbF9kZiA9IHJlYWQuY3N2KHBhc3RlMChyZXRlc3RfZGF0YV9wYXRoLCAndHJpYWxfbnVtX3JlbF9kZl90YnQuY3N2JykpCgpjb2xzIDwtIGMoIkFjY3VyYWN5IiA9ICcjMDg0NTk0JywgIkF2ZXJhZ2UgUlQgZXJyb3IiID0gJyM5OTAwMGQnLCAiU0QgUlQgZXJyb3IiID0gJyNjYjE4MWQnLCAiQXZlcmFnZSBSVCBjb3JyZWN0IiA9ICcjZWYzYjJjJywgIlNEIFJUIGNvcnJlY3QiID0gJyNmYjZhNGEnLCAiTWlzc2VkIHBlcmNlbnRhZ2UiID0gJyMyMTcxYjUnLCAiQ3VlIHN3aXRjaCBjb3N0IFJUICgxMDApIiA9ICcjZmM5MjcyJywgIkN1ZSBzd2l0Y2ggY29zdCBSVCAoOTAwKSIgPSAnI2ZjYmJhMScsICJUYXNrIHN3aXRjaCBjb3N0IFJUICgxMDApIiA9ICcjZmVlMGQyJywgIlRhc2sgc3dpdGNoIGNvc3QgUlQgKDkwMCkiID0gJyNmZmY1ZjAnLCAiQ3VlIHN3aXRjaCBjb3N0IEFjYyAoMTAwKSIgPSAnIzQyOTJjNicsICJDdWUgc3dpdGNoIGNvc3QgQWNjICg5MDApIiA9ICcjNmJhZWQ2JywgIlRhc2sgc3dpdGNoIGNvc3QgQWNjICgxMDApIiA9ICcjOWVjYWUxJywgIlRhc2sgc3dpdGNoIGNvc3QgQWNjICg5MDApIiA9ICcjYzZkYmVmJykKCnRyaWFsX251bV9yZWxfZGYgJT4lCiAgZ2F0aGVyKGtleSwgdmFsdWUsIC1icmVha3MpICU+JQogIGdncGxvdChhZXMoYnJlYWtzKjEwLCB2YWx1ZSwgY29sb3I9ZmFjdG9yKGtleSwgbGV2ZWxzID0gYygiYWNjX2ljYyIsICJhdmdfcnRfZXJyb3JfaWNjIiwgInN0ZF9ydF9lcnJvcl9pY2MiLCAiYXZnX3J0X2ljYyIsICJzdGRfcnRfaWNjIiwgIm1pc3NlZF9wZXJjZW50X2ljYyIsICJjdWVfc3dpdGNoX2Nvc3RfcnRfMTAwX2ljYyIsICJjdWVfc3dpdGNoX2Nvc3RfcnRfOTAwX2ljYyIsICJ0YXNrX3N3aXRjaF9jb3N0X3J0XzEwMF9pY2MiLCAidGFza19zd2l0Y2hfY29zdF9ydF85MDBfaWNjIiwgImN1ZV9zd2l0Y2hfY29zdF9hY2NfMTAwX2ljYyIsICJjdWVfc3dpdGNoX2Nvc3RfYWNjXzkwMF9pY2MiLCAidGFza19zd2l0Y2hfY29zdF9hY2NfMTAwX2ljYyIsICJ0YXNrX3N3aXRjaF9jb3N0X2FjY185MDBfaWNjIiksIGxhYmVscyA9IGMoIkFjY3VyYWN5IiwgIkF2ZXJhZ2UgUlQgZXJyb3IiLCAiU0QgUlQgZXJyb3IiLCAiQXZlcmFnZSBSVCBjb3JyZWN0IiwgIlNEIFJUIGNvcnJlY3QiLCAiTWlzc2VkIHBlcmNlbnRhZ2UiLCAiQ3VlIHN3aXRjaCBjb3N0IFJUICgxMDApIiwgIkN1ZSBzd2l0Y2ggY29zdCBSVCAoOTAwKSIsICJUYXNrIHN3aXRjaCBjb3N0IFJUICgxMDApIiwgIlRhc2sgc3dpdGNoIGNvc3QgUlQgKDkwMCkiLCAiQ3VlIHN3aXRjaCBjb3N0IEFjYyAoMTAwKSIsICJDdWUgc3dpdGNoIGNvc3QgQWNjICg5MDApIiwgIlRhc2sgc3dpdGNoIGNvc3QgQWNjICgxMDApIiwgIlRhc2sgc3dpdGNoIGNvc3QgQWNjICg5MDApIikpKSkrCiAgZ2VvbV9wb2ludCgpKwogIGdlb21fbGluZSgpKwogIHRoZW1lX2J3KCkrCiAgeGxhYigiTnVtYmVyIG9mIHRyaWFscyIpKwogIHlsYWIoIklDQyIpKwogIHRoZW1lKGxlZ2VuZC50aXRsZSA9IGVsZW1lbnRfYmxhbmsoKSwKICAgICAgICBsZWdlbmQucG9zaXRpb24gPSAnYm90dG9tJykrCiAgc2NhbGVfY29sb3JfbWFudWFsKHZhbHVlcyA9IGNvbHMsCiAgICAgICAgICAgICAgICAgICAgIGJyZWFrcyA9IGMoIkF2ZXJhZ2UgUlQgZXJyb3IiLCAiQWNjdXJhY3kiLCAiU0QgUlQgZXJyb3IiLCAiTWlzc2VkIHBlcmNlbnRhZ2UiLCAiQXZlcmFnZSBSVCBjb3JyZWN0IiwgIkN1ZSBzd2l0Y2ggY29zdCBBY2MgKDEwMCkiLCAiU0QgUlQgY29ycmVjdCIsICJDdWUgc3dpdGNoIGNvc3QgQWNjICg5MDApIiwgIkN1ZSBzd2l0Y2ggY29zdCBSVCAoMTAwKSIsICJUYXNrIHN3aXRjaCBjb3N0IEFjYyAoMTAwKSIsICJUYXNrIHN3aXRjaCBjb3N0IFJUICgxMDApIiwgIlRhc2sgc3dpdGNoIGNvc3QgQWNjICg5MDApIiwgIlRhc2sgc3dpdGNoIGNvc3QgUlQgKDkwMCkiKSkrCiAgeWxpbSgtMSwxKSsKICBndWlkZXMoY29sb3IgPSBndWlkZV9sZWdlbmQobmNvbD0yLCBieXJvdz1UKSkKCmdnc2F2ZSgnSW50cmFtZWFzdXJlX1RyaWFsbnVtX0RlcGVuZGVuZGVuY2UuanBnJywgZGV2aWNlID0gImpwZWciLCBwYXRoID0gIi9Vc2Vycy96ZXluZXBlbmthdmkvRHJvcGJveC9Qb2xkcmFja0xhYi9TUk9fUmV0ZXN0X0FuYWx5c2VzL291dHB1dC9maWd1cmVzIiwgd2lkdGggPSAxMCwgaGVpZ2h0ID0gNSwgdW5pdHMgPSAiaW4iLCBkcGkgPSAxMDApCmBgYAoKIyMjIFJhdyB2cyBERE0KCkNoZWNraW5nIERETSByZXN1bHRzIGluIHRoZSBib290c3RyYXBwZWQgZXN0aW1hdGVzLiBWYXJpYWJsZXMgdXNpbmcgYWxsIHRyaWFscyBhcmUgc2lnbmlmaWNhbnRseSBtb3JlIHJlbGlhYmxlIGNvbXBhcmVkIHRvIGRpZmZlcmVuY2Ugc2NvcmVzLiBSYXcgbWVhc3VyZXMgZG9uJ3QgZGlmZmVyIGZyb20gRERNIHBhcmFtZXRlcnMuIFdoaWNoIERETSBpcyBiZXR0ZXIgZGVwZW5kcyBvbiB3aGV0aGVyIGFsbCB0cmlhbHMgYXJlIHVzZWQuCgpgYGB7ciB3YXJuaW5nPUZBTFNFLCBtZXNzYWdlPUZBTFNFfQp0bXAgPSBtZWFzdXJlX2xhYmVscyAlPiUKICBtdXRhdGUoZHYgPSBhcy5jaGFyYWN0ZXIoZHYpKSAlPiUKICBmaWx0ZXIoZGRtX3Rhc2sgPT0gMSwgCiAgICAgICAgIG92ZXJhbGxfZGlmZmVyZW5jZSAhPSAiY29uZGl0aW9uIiwKICAgICAgICAgcnRfYWNjICE9ICdvdGhlcicpICU+JQogIGRyb3BfbmEoKSAlPiUKICBsZWZ0X2pvaW4oYm9vdF9kZlssYygiZHYiLCAiaWNjIiwgInNwZWFybWFuIildLCBieSA9ICdkdicpCgp0bXAgJT4lCiAgZHJvcF9uYSgpICU+JSAjdHJ5IHJlbW92aW5nIHRoaXMgaW4gZmluYWwgcmVsZWFzZQogIGdyb3VwX2J5KG92ZXJhbGxfZGlmZmVyZW5jZSwgcmF3X2ZpdCwgcnRfYWNjKSAlPiUKICBzdW1tYXJpc2UoaWNjX21lZGlhbiA9IHF1YW50aWxlKGljYywgcHJvYnMgPSAwLjUpLAogICAgICAgICAgICBpY2NfMi41ID0gcXVhbnRpbGUoaWNjLCBwcm9icyA9IDAuMDI1KSwKICAgICAgICAgICAgaWNjXzk3LjUgPSBxdWFudGlsZShpY2MsIHByb2JzID0gMC45NzUpLAogICAgICAgICAgICBzcGVhcm1hbl9tZWRpYW4gPSBxdWFudGlsZShzcGVhcm1hbiwgcHJvYnMgPSAwLjUpLAogICAgICAgICAgICBzcGVhcm1hbl8yLjUgPSBxdWFudGlsZShzcGVhcm1hbiwgcHJvYnMgPSAwLjAyNSksCiAgICAgICAgICAgIHNwZWFybWFuXzk3LjUgPSBxdWFudGlsZShzcGVhcm1hbiwgcHJvYnMgPSAwLjk3NSksCiAgICAgICAgICAgIG51bV92YXJzID0gbigpLzEwMDApICU+JQogIGRhdGF0YWJsZSgpICU+JQogIGZvcm1hdFJvdW5kKGNvbHVtbnM9YygnaWNjX21lZGlhbicsICdpY2NfMi41JywgJ2ljY185Ny41JywgJ3NwZWFybWFuX21lZGlhbicsICdzcGVhcm1hbl8yLjUnLCAnc3BlYXJtYW5fOTcuNScpLCBkaWdpdHM9MykKYGBgCgpgYGB7cn0KdG1wX3NhdmUgPSB0bXAgJT4lCiAgZHJvcF9uYSgpICU+JSAjdHJ5IHJlbW92aW5nIHRoaXMgaW4gZmluYWwgcmVsZWFzZQogIGdyb3VwX2J5KG92ZXJhbGxfZGlmZmVyZW5jZSwgcmF3X2ZpdCwgcnRfYWNjKSAlPiUKICBzdW1tYXJpc2UoaWNjX21lZGlhbiA9IHF1YW50aWxlKGljYywgcHJvYnMgPSAwLjUpLAogICAgICAgICAgICBpY2NfMi41ID0gcXVhbnRpbGUoaWNjLCBwcm9icyA9IDAuMDI1KSwKICAgICAgICAgICAgaWNjXzk3LjUgPSBxdWFudGlsZShpY2MsIHByb2JzID0gMC45NzUpLAogICAgICAgICAgICBudW1fdmFycyA9IG4oKS8xMDAwKSAlPiUKICB1bmdyb3VwKCkgJT4lCiAgbXV0YXRlKG92ZXJhbGxfZGlmZmVyZW5jZSA9IGFzLmNoYXJhY3RlcihvdmVyYWxsX2RpZmZlcmVuY2UpLAogICAgICAgICByYXdfZml0ID0gYXMuY2hhcmFjdGVyKHJhd19maXQpLAogICAgICAgICBydF9hY2MgPSBhcy5jaGFyYWN0ZXIocnRfYWNjKSkgJT4lCiAgYXJyYW5nZSgtaWNjX21lZGlhbikKCnNqdC5kZih0bXBfc2F2ZSAlPiUgbXV0YXRlX2lmKGlzLm51bWVyaWMsIGZ1bnMocm91bmQoLiwgMykpKSwgZGVzY3JpYmU9RiwgaGlkZS5wcm9ncmVzcyA9IFRSVUUsIHNob3cucm93bmFtZXMgPSBGQUxTRSwgZmlsZSA9ICIvVXNlcnMvemV5bmVwZW5rYXZpL0Ryb3Bib3gvUG9sZHJhY2tMYWIvU1JPX1JldGVzdF9BbmFseXNlcy9vdXRwdXQvdGFibGVzL2RkbV9yZWxfdGFibGUuZG9jIikKYGBgCgpDb21wYXJpbmcgb3ZlcmFsbCB2cyBkaWZmZXJlbmNlOiBvdmVyYWxsIGhhcyBoaWdoZXIgcmVsaWFiaWxpdHkgdGhhbiBkaWZmZXJlbmNlLgoKYGBge3J9CnN1bW1hcnkobG1lclRlc3Q6OmxtZXIoaWNjIH4gb3ZlcmFsbF9kaWZmZXJlbmNlICsgKDF8ZHYpICx0bXApKQpgYGAKCiJjb25zaXN0ZW50IHdpdGggdGhlIHN1bW1pbmcgb2YgdmFyaWFuY2UgaW4gdGhlIGRpZmZlcmVuY2Ugc2NvcmUiCgpgYGB7cn0KdG1wID0gbWVhc3VyZV9sYWJlbHMgJT4lCiAgbXV0YXRlKGR2ID0gYXMuY2hhcmFjdGVyKGR2KSkgJT4lCiAgZmlsdGVyKGRkbV90YXNrID09IDEsIAogICAgICAgICBvdmVyYWxsX2RpZmZlcmVuY2UgIT0gImNvbmRpdGlvbiIsCiAgICAgICAgIHJ0X2FjYyAhPSAnb3RoZXInKSAlPiUKICBkcm9wX25hKCkgJT4lCiAgbGVmdF9qb2luKGJvb3RfZGZbLGMoImR2IiwgImljYyIsICJ2YXJfc3VicyIsInZhcl9pbmQiLCAidmFyX3Jlc2lkIildLCBieSA9ICdkdicpICU+JQogIG11dGF0ZSh2YXJfc3Vic19wY3QgPSB2YXJfc3Vicy8odmFyX3N1YnMrdmFyX2luZCt2YXJfcmVzaWQpLAogICAgICAgICB2YXJfaW5kX3BjdCA9IHZhcl9pbmQvKHZhcl9zdWJzK3Zhcl9pbmQrdmFyX3Jlc2lkKSwgCiAgICAgICAgIHZhcl9yZXNpZF9wY3QgPSB2YXJfcmVzaWQvKHZhcl9zdWJzK3Zhcl9pbmQrdmFyX3Jlc2lkKSkgJT4lCiAgc2VsZWN0KC10YXNrLCAtZGRtX3Rhc2ssIC1udW1fYWxsX3RyaWFscywgLXZhcl9yZXNpZCwgLXZhcl9zdWJzLCAtdmFyX2luZCkgJT4lCiAgZ2F0aGVyKGtleSwgdmFsdWUsIC1kdiwgLW92ZXJhbGxfZGlmZmVyZW5jZSwgLXJhd19maXQsIC1ydF9hY2MsIC1pY2MpCgp0bXAgJT4lCiAgZmlsdGVyKGtleSA9PSAidmFyX3Jlc2lkX3BjdCIpICU+JQogIGdncGxvdChhZXMocmF3X2ZpdCwgdmFsdWUsIGZpbGw9b3ZlcmFsbF9kaWZmZXJlbmNlKSkrCiAgZ2VvbV9ib3hwbG90KCkKICAKc3VtbWFyeShsbWVyKHZhbHVlICB+IG92ZXJhbGxfZGlmZmVyZW5jZSpyYXdfZml0ICsgKDF8ZHYpLCBkYXRhPXRtcCAlPiUgZmlsdGVyKGtleT09InZhcl9yZXNpZF9wY3QiKSkpCgojU2ltcGxlIGVmZmVjdHMKc3VtbWFyeShsbWVyKHZhbHVlICB+IG92ZXJhbGxfZGlmZmVyZW5jZSArICgxfGR2KSwgZGF0YT10bXAgJT4lIGZpbHRlcihrZXk9PSJ2YXJfcmVzaWRfcGN0IiwgcmF3X2ZpdD09ICJFWiIpKSkKCnN1bW1hcnkobG1lcih2YWx1ZSAgfiBvdmVyYWxsX2RpZmZlcmVuY2UgKyAoMXxkdiksIGRhdGE9dG1wICU+JSBmaWx0ZXIoa2V5PT0idmFyX3Jlc2lkX3BjdCIsIHJhd19maXQ9PSAicmF3IikpKQoKc3VtbWFyeShsbWVyKHZhbHVlICB+IG92ZXJhbGxfZGlmZmVyZW5jZSArICgxfGR2KSwgZGF0YT10bXAgJT4lIGZpbHRlcihrZXk9PSJ2YXJfcmVzaWRfcGN0IiwgcmF3X2ZpdD09ICJoZGRtIikpKQpgYGAKCkNoZWNraW5nIFJvZ29zYSdzIGNsYWltIHRoYXQg4oCYRGlmZmVyZW5jZSBzY29yZXMgYXJlIHJlbGlhYmxlIHdoZW4gaW5kaXZpZHVhbCBkaWZmZXJlbmNlcyBpbiB0cnVlIGNoYW5nZSBleGlzdC7igJkKSG93IGRvIHdlIG1lYXN1cmUg4oCYaW5kaXZpZHVhbCBkaWZmZXJlbmNlIGluIHRydWUgY2hhbmdl4oCZIAoxLmJldHdlZW4gc3ViamVjdCB2YXJpYWJpbGl0eSBpbiBkaWZmZXJlbmNlIHNjb3JlIGRpc3RyaWJ1dGlvbnMKMi53aXRoaW4gc3ViamVjdCB2YXJpYW5jZSBpbiBpY2MKVXNpbmcgMiAoYmVjYXVzZSBjb21wYXJhYmxlIGJldHdlZW4gc3ViamVjdCB2YXJpYW5jZSBhY3Jvc3MgZGlmZmVyZW50IG1lYXN1cmVzIHdvdWxkIGJlIGhhcmQpCkV4cGVjdGluZyB0byBzZWU6IHBvc2l0aXZlIGNvcnJlbGF0aW9uIGJldHdlZW4gbWVhbiBpY2MgYW5kIG1lYW4gdmFyX2luZF9wY3QKUmVzdWx0OiBUaGUgY29ycmVsYXRpb24gYmV0d2VlbiBtZWFuIGljYyBhbmQgbWVhbiB2YXIgaW5kIHBjdCBpcyBOT1Qgc2lnbmlmaWNhbnQgQlVUIGxvb2tpbmcgYXQgdGhlIGRpc3RyaWJ1dGlvbiBvZiBtZWFuIHZhciBpbmQgcGN0IHRoZXJlIGlzIG5vdCBhIGxvdCBvZiB2YXJpYWJpbGl0eSB0CgpgYGB7cn0KdG1wID0gdG1wICU+JQogIHNwcmVhZChrZXksIHZhbHVlKSAlPiUKICBncm91cF9ieShkdikgJT4lCiAgc3VtbWFyaXNlKG1lYW5fdmFyX2luZF9wY3QgPSBtZWFuKHZhcl9pbmRfcGN0KSwKICAgICAgICAgICAgbWVhbl92YXJfc3Vic19wY3QgPSBtZWFuKHZhcl9zdWJzX3BjdCksCiAgICAgICAgICAgIG1lYW5faWNjID0gbWVhbihpY2MpKSAlPiUKICBsZWZ0X2pvaW4obWVhc3VyZV9sYWJlbHMsIGJ5PSdkdicpCgp0bXAgJT4lCiAgZmlsdGVyKG92ZXJhbGxfZGlmZmVyZW5jZSA9PSAnZGlmZmVyZW5jZScpICU+JQogIGdncGxvdChhZXMobWVhbl92YXJfaW5kX3BjdCwgbWVhbl9pY2MpKSsKICBnZW9tX3BvaW50KCkrCiAgZ2VvbV9hYmxpbmUoc2xvcGU9MSwgaW50ZXJjZXB0PTApKwogIGdlb21fc21vb3RoKG1ldGhvZCA9ICJsbSIpCgpzdW1tYXJ5KGxtKG1lYW5faWNjIH4gbWVhbl92YXJfaW5kX3BjdCxkYXRhPXRtcCAlPiUKICBmaWx0ZXIob3ZlcmFsbF9kaWZmZXJlbmNlID09ICdkaWZmZXJlbmNlJykpKQoKdG1wICU+JQogIGZpbHRlcihvdmVyYWxsX2RpZmZlcmVuY2UgPT0gJ2RpZmZlcmVuY2UnKSAlPiUKICBnZ3Bsb3QoYWVzKG1lYW5fdmFyX2luZF9wY3QpKSsKICBnZW9tX2RlbnNpdHkoKQoKc3VtbWFyeSh0bXAkbWVhbl92YXJfaW5kX3BjdFt0bXAkb3ZlcmFsbF9kaWZmZXJlbmNlID09ICJkaWZmZXJlbmNlIl0pCnN1bW1hcnkodG1wJG1lYW5fdmFyX3N1YnNfcGN0W3RtcCRvdmVyYWxsX2RpZmZlcmVuY2UgPT0gImRpZmZlcmVuY2UiXSkKCnRtcCA9IHRtcCAlPiUgZmlsdGVyKG92ZXJhbGxfZGlmZmVyZW5jZSA9PSAnZGlmZmVyZW5jZScpCnQudGVzdCh0bXAkbWVhbl92YXJfaW5kX3BjdCwgdG1wJG1lYW5fdmFyX3N1YnNfcGN0LCBwYWlyZWQ9VCkKYGBgCgpgYGB7cn0KdG1wID0gbWVhc3VyZV9sYWJlbHMgJT4lCiAgbXV0YXRlKGR2ID0gYXMuY2hhcmFjdGVyKGR2KSkgJT4lCiAgZmlsdGVyKGRkbV90YXNrID09IDEsIAogICAgICAgICBvdmVyYWxsX2RpZmZlcmVuY2UgIT0gImNvbmRpdGlvbiIsCiAgICAgICAgIHJ0X2FjYyAhPSAnb3RoZXInKSAlPiUKICBkcm9wX25hKCkgJT4lCiAgbGVmdF9qb2luKGJvb3RfZGZbLGMoImR2IiwgImljYyIsICJzcGVhcm1hbiIpXSwgYnkgPSAnZHYnKQpgYGAKCkNvbXBhcmluZyByYXcgdnMgZGRtIGluIG92ZXJhbGwgZXN0aW1hdGVzOiBFWiBpcyBzaWduaWZpY2FudGx5IGJldHRlciB0aGFuIEhERE0gYW5kIGNvbXBhcmFibGUgdG8gcmF3IGVzdGltYXRlcy4KCmBgYHtyfQpzdW1tYXJ5KGxtZXJUZXN0OjpsbWVyKGljYyB+IHJhd19maXQgKyAoMXxkdikgLHRtcCAlPiUgZmlsdGVyKG92ZXJhbGxfZGlmZmVyZW5jZSA9PSAib3ZlcmFsbCIpKSkKYGBgCgpDb21wYXJpbmcgcmF3IHZzIGRkbSBpbiBkaWZmZXJlbmNlIHNjb3JlczogRVogaXMgc2lnbmlmaWNhbnRseSB3b3JzZSB0aGFuIEhERE0gYW5kIGNvbXBhcmFibGUgdG8gcmF3IGVzdGltYXRlcy4KCmBgYHtyfQpzdW1tYXJ5KGxtZXJUZXN0OjpsbWVyKGljYyB+IHJhd19maXQgKyAoMXxkdikgLHRtcCAlPiUgZmlsdGVyKG92ZXJhbGxfZGlmZmVyZW5jZSA9PSAiZGlmZmVyZW5jZSIpKSkKYGBgCgpgYGB7cn0KdG1wICU+JQogIGdncGxvdChhZXMoZmFjdG9yKHJhd19maXQsIGxldmVscyA9IGMoInJhdyIsICJFWiIsICJoZGRtIiksIGxhYmVscz1jKCJSYXciLCAiRVotZGlmZnVzaW9uIiwgIkhpZXJhcmNoaWNhbCBkaWZmdXNpb24iKSksIGljYywgZmlsbD1mYWN0b3IocnRfYWNjLCBsZXZlbHMgPSBjKCJydCIsImFjY3VyYWN5IiwgImRyaWZ0IHJhdGUiLCAidGhyZXNob2xkIiwgIm5vbi1kZWNpc2lvbiIpLCBsYWJlbHM9YygiUmVzcG9uc2UgVGltZSIsICJBY2N1cmFjeSIsIkRyaWZ0IFJhdGUiLCAiVGhyZXNob2xkIiwgIk5vbi1kZWNpc2lvbiIpKSkpKwogIGdlb21fYm94cGxvdCgpKwogIGZhY2V0X3dyYXAofmZhY3RvcihvdmVyYWxsX2RpZmZlcmVuY2UsIGxldmVscz1jKCJvdmVyYWxsIiwgImRpZmZlcmVuY2UiKSwgbGFiZWxzPWMoIk92ZXJhbGwiLCAiRGlmZmVyZW5jZSIpKSkrCiAgdGhlbWVfYncoKSsKICB5bGFiKCJJQ0MiKSsKICB4bGFiKCIiKSsKICB0aGVtZShsZWdlbmQudGl0bGUgPSBlbGVtZW50X2JsYW5rKCksCiAgICAgICAgbGVnZW5kLnBvc2l0aW9uID0gJ2JvdHRvbScsCiAgICAgICAgbGVnZW5kLnRleHQgPSBlbGVtZW50X3RleHQoc2l6ZT0xNiksCiAgICAgICAgc3RyaXAudGV4dCA9IGVsZW1lbnRfdGV4dChzaXplPTE2KSwKICAgICAgICBheGlzLnRleHQgPSBlbGVtZW50X3RleHQoc2l6ZSA9IDE2KSwKICAgICAgICB0ZXh0ID0gZWxlbWVudF90ZXh0KHNpemU9MTYpKSsKICBndWlkZXMoZmlsbCA9IGd1aWRlX2xlZ2VuZChuY29sID0gMikpKwogIHNjYWxlX2ZpbGxfZGlzY3JldGUoYnJlYWtzPWMoIkRyaWZ0IFJhdGUiLCAiVGhyZXNob2xkIiwgIk5vbi1kZWNpc2lvbiIsICJSZXNwb25zZSBUaW1lIiwgIkFjY3VyYWN5IikpCgpnZ3NhdmUoJ0Jvb3RzdHJhcF9ERE1fQ29tcC5qcGcnLCBkZXZpY2UgPSAianBlZyIsIHBhdGggPSAiL1VzZXJzL3pleW5lcGVua2F2aS9Ecm9wYm94L1BvbGRyYWNrTGFiL1NST19SZXRlc3RfQW5hbHlzZXMvb3V0cHV0L2ZpZ3VyZXMvIiwgd2lkdGggPSAxNCwgaGVpZ2h0ID0gMTAsIHVuaXRzID0gImluIiwgbGltaXRzaXplID0gRkFMU0UpCmBgYAoKYGBge3J9CnRtcDIgPSBtZWFzdXJlX2xhYmVscyAlPiUKICBtdXRhdGUoZHYgPSBhcy5jaGFyYWN0ZXIoZHYpKSAlPiUKICBmaWx0ZXIoZGRtX3Rhc2sgPT0gMSwgCiAgICAgICAgIG92ZXJhbGxfZGlmZmVyZW5jZSAhPSAiY29uZGl0aW9uIiwKICAgICAgICAgcnRfYWNjICE9ICJvdGhlciIpICU+JQogIGRyb3BfbmEoKSAlPiUKICBsZWZ0X2pvaW4ocmVsX2RmWyxjKCJkdiIsICJpY2MiLCAic3BlYXJtYW4iKV0sIGJ5ID0gJ2R2JykKCnRtcDIgJT4lCiAgZ2dwbG90KGFlcyhmYWN0b3IocmF3X2ZpdCwgbGV2ZWxzID0gYygicmF3IiwgIkVaIiwgImhkZG0iKSwgbGFiZWxzPWMoIlJhdyIsICJFWi1kaWZmdXNpb24iLCAiSGllcmFyY2hpY2FsIGRpZmZ1c2lvbiIpKSwgaWNjLCBmaWxsPWZhY3RvcihydF9hY2MsIGxldmVscyA9IGMoInJ0IiwiYWNjdXJhY3kiLCAiZHJpZnQgcmF0ZSIsICJ0aHJlc2hvbGQiLCAibm9uLWRlY2lzaW9uIiksIGxhYmVscz1jKCJSZXNwb25zZSBUaW1lIiwgIkFjY3VyYWN5IiwiRHJpZnQgUmF0ZSIsICJUaHJlc2hvbGQiLCAiTm9uLWRlY2lzaW9uIikpKSkrCiAgZ2VvbV9ib3hwbG90KCkrCiAgZmFjZXRfd3JhcCh+ZmFjdG9yKG92ZXJhbGxfZGlmZmVyZW5jZSwgbGV2ZWxzPWMoIm92ZXJhbGwiLCAiZGlmZmVyZW5jZSIpLCBsYWJlbHM9YygiT3ZlcmFsbCIsICJEaWZmZXJlbmNlIikpKSsKICB0aGVtZV9idygpKwogIHlsYWIoIklDQyIpKwogIHhsYWIoIiIpKwogIHRoZW1lKGxlZ2VuZC50aXRsZSA9IGVsZW1lbnRfYmxhbmsoKSwKICAgICAgICBsZWdlbmQucG9zaXRpb24gPSAnYm90dG9tJywKICAgICAgICBsZWdlbmQudGV4dCA9IGVsZW1lbnRfdGV4dChzaXplPTE2KSwKICAgICAgICBzdHJpcC50ZXh0ID0gZWxlbWVudF90ZXh0KHNpemU9MTYpLAogICAgICAgIGF4aXMudGV4dCA9IGVsZW1lbnRfdGV4dChzaXplID0gMTYpLAogICAgICAgIHRleHQgPSBlbGVtZW50X3RleHQoc2l6ZT0xNikpKwogIGd1aWRlcyhmaWxsID0gZ3VpZGVfbGVnZW5kKG5jb2wgPSAyKSkrCiAgc2NhbGVfZmlsbF9kaXNjcmV0ZShicmVha3M9YygiRHJpZnQgUmF0ZSIsICJUaHJlc2hvbGQiLCAiTm9uLWRlY2lzaW9uIiwgIlJlc3BvbnNlIFRpbWUiLCAiQWNjdXJhY3kiKSkKCmdnc2F2ZSgnUG9pbnRFc3RfRERNX0NvbXAuanBnJywgZGV2aWNlID0gImpwZWciLCBwYXRoID0gIi9Vc2Vycy96ZXluZXBlbmthdmkvRHJvcGJveC9Qb2xkcmFja0xhYi9TUk9fUmV0ZXN0X0FuYWx5c2VzL291dHB1dC9maWd1cmVzLyIsIHdpZHRoID0gMTQsIGhlaWdodCA9IDEwLCB1bml0cyA9ICJpbiIsIGxpbWl0c2l6ZSA9IEZBTFNFKQpgYGAKCiMjIFN1cnZleSByZWxpYWJpbGl0aWVzCgpgYGB7ciB3YXJuaW5nPUZBTFNFLCBtZXNzYWdlPUZBTFNFfQp0bXAgPSBtZWFzdXJlX2xhYmVscyAlPiUKICBtdXRhdGUoZHYgPSBhcy5jaGFyYWN0ZXIoZHYpKSAlPiUKICBmaWx0ZXIodGFzayA9PSAnc3VydmV5JykgJT4lCiAgbGVmdF9qb2luKGJvb3RfZGZbLGMoImR2IiwgImljYyIsICJzcGVhcm1hbiIpXSwgYnkgPSAnZHYnKSAlPiUKICBmaWx0ZXIob3ZlcmFsbF9kaWZmZXJlbmNlICE9ICdkaWZmZXJlbmNlJyAmIHJhd19maXQgJWluJSBjKCdFWicsICdoZGRtJykgPT0gRkFMU0UpJT4lCiAgc2VwYXJhdGUoZHYsIGMoJ3Rhc2tfbmFtZScsICdleHRyYV8xJywgJ2V4dHJhXzInKSwgc2VwID0gJ1xcLicscmVtb3ZlPUZBTFNFKSAlPiUKICBzZWxlY3QoLWV4dHJhXzEsIC1leHRyYV8yKSAlPiUKICBncm91cF9ieSh0YXNrX25hbWUpICU+JQogIHN1bW1hcmlzZShtZWRpYW5faWNjID0gbWVkaWFuKGljYyksCiAgICAgICAgICAgIG1lZGlhbl9zcGVhcm1hbiA9IG1lZGlhbihzcGVhcm1hbiksCiAgICAgICAgICAgIGljY18yLjUgPSBxdWFudGlsZShpY2MsIHByb2JzID0gMC4wMjUpLAogICAgICAgICAgICBpY2NfOTcuNSA9IHF1YW50aWxlKGljYywgcHJvYnMgPSAwLjk3NSksCiAgICAgICAgICAgIG1pbl9zcGVhcm1hbiA9IG1pbihzcGVhcm1hbiksCiAgICAgICAgICAgIG1heF9zcGVhcm1hbiA9IG1heChzcGVhcm1hbiksCiAgICAgICAgICAgIG51bV9tZWFzdXJlcyA9IG4oKS8xMDAwLAogICAgICAgICAgICBtZWFuX251bV90cmlhbHMgPSByb3VuZChtZWFuKG51bV9hbGxfdHJpYWxzKSkpJT4lCiAgYXJyYW5nZSgtbWVkaWFuX2ljYywgLW1lZGlhbl9zcGVhcm1hbikKCnRtcCAlPiUKICBkYXRhdGFibGUoKSAlPiUKICBmb3JtYXRSb3VuZChjb2x1bW5zPWMoJ21lZGlhbl9zcGVhcm1hbicsICdtZWRpYW5faWNjJywKICAgICAgICAgICAgICAgICAgICAgICAgJ21pbl9zcGVhcm1hbicsICdpY2NfMi41JywKICAgICAgICAgICAgICAgICAgICAgICAgJ21heF9zcGVhcm1hbicsICdpY2NfOTcuNScpLCBkaWdpdHM9MykKYGBgCgpgYGB7cn0KdG1wID0gdG1wJT4lCiAgc2VsZWN0KC1taW5fc3BlYXJtYW4sIC1tYXhfc3BlYXJtYW4sLW1lZGlhbl9zcGVhcm1hbikgJT4lCiAgbXV0YXRlKHRhc2tfbmFtZSA9IGdzdWIoIl8iLCAiICIsIHRhc2tfbmFtZSksCiAgICAgICAgIHRhc2tfbmFtZSA9IGdzdWIoIihefFtbOnNwYWNlOl1dKShbWzphbHBoYTpdXSkiLCAiXFwxXFxVXFwyIiwgdGFza19uYW1lLCBwZXJsPVRSVUUpLAogICAgICAgICB0YXNrX25hbWUgPSBnc3ViKCJTdXJ2ZXkiLCAiIiwgdGFza19uYW1lKSkKCm5hbWVzKHRtcCkgPSBnc3ViKCJfIiwgIiAiLCBuYW1lcyh0bXApKQpuYW1lcyh0bXApID0gZ3N1YigiKF58W1s6c3BhY2U6XV0pKFtbOmFscGhhOl1dKSIsICJcXDFcXFVcXDIiLCBuYW1lcyh0bXApLCBwZXJsPVRSVUUpCgpzanQuZGYodG1wICU+JSBtdXRhdGVfaWYoaXMubnVtZXJpYywgZnVucyhyb3VuZCguLCAzKSkpLCBkZXNjcmliZT1GLCBoaWRlLnByb2dyZXNzID0gVFJVRSwgc2hvdy5yb3duYW1lcyA9IEZBTFNFLCBmaWxlID0gIi9Vc2Vycy96ZXluZXBlbmthdmkvRHJvcGJveC9Qb2xkcmFja0xhYi9TUk9fUmV0ZXN0X0FuYWx5c2VzL291dHB1dC90YWJsZXMvc3VydmV5X3JlbF90YWJsZS5kb2MiKQpgYGAKCiMjIyBOdW1iZXIgb2YgaXRlbXMKCkRvZXMgbnVtYmVyIG9mIGl0ZW1zIGluIGEgc3VydmV5IGhhdmUgYSBzaWduaWZpY2FudCBlZmZlY3Qgb24gdGhlIGF2ZXJhZ2UgSUNDIG9mIHN1cnZleSBtZWFzdXJlcz8gTm8uIAoKYGBge3Igd2FybmluZz1GQUxTRSwgbWVzc2FnZT1GQUxTRX0KdG1wID0gbWVhc3VyZV9sYWJlbHMgJT4lCiAgbXV0YXRlKGR2ID0gYXMuY2hhcmFjdGVyKGR2KSkgJT4lCiAgZmlsdGVyKHRhc2sgPT0gJ3N1cnZleScpICU+JQogICMgbGVmdF9qb2luKHJlbF9kZlssYygiZHYiLCAic3BlYXJtYW4iLCJpY2MiKV0sIGJ5PSdkdicpICU+JQogIGxlZnRfam9pbihib290X2RmWyxjKCJkdiIsICJzcGVhcm1hbiIsImljYyIpXSwgYnk9J2R2JykgJT4lCiAgZmlsdGVyKG92ZXJhbGxfZGlmZmVyZW5jZSAhPSAnZGlmZmVyZW5jZScgJiByYXdfZml0ICVpbiUgYygnRVonLCAnaGRkbScpID09IEZBTFNFKSU+JQogIHNlcGFyYXRlKGR2LCBjKCd0YXNrX25hbWUnLCAnZXh0cmFfMScsICdleHRyYV8yJyksIHNlcCA9ICdcXC4nLHJlbW92ZT1GQUxTRSkgJT4lCiAgc2VsZWN0KC1leHRyYV8xLCAtZXh0cmFfMikKCiMgc3VtbWFyeShsbShpY2MgfiBudW1fYWxsX3RyaWFscywgZGF0YSA9IHRtcCkpCnN1bW1hcnkobG1lclRlc3Q6OmxtZXIoaWNjIH4gbnVtX2FsbF90cmlhbHMgKyAoMXxkdiksIGRhdGEgPSB0bXApKQpgYGAKCmBgYHtyIHdhcm5pbmc9RkFMU0UsIG1lc3NhZ2U9RkFMU0V9Cm1lYXN1cmVfbGFiZWxzICU+JQogIG11dGF0ZShkdiA9IGFzLmNoYXJhY3RlcihkdikpICU+JQogIGZpbHRlcih0YXNrID09ICd0YXNrJykgJT4lCiAgIyBsZWZ0X2pvaW4ocmVsX2RmWyxjKCJkdiIsICJzcGVhcm1hbiIsImljYyIpXSwgYnk9J2R2JykgJT4lCiAgbGVmdF9qb2luKGJvb3RfZGZbLGMoImR2IiwgInNwZWFybWFuIiwiaWNjIildLCBieT0nZHYnKSAlPiUKICBmaWx0ZXIob3ZlcmFsbF9kaWZmZXJlbmNlICE9ICdkaWZmZXJlbmNlJyAmIHJhd19maXQgJWluJSBjKCdFWicsICdoZGRtJykgPT0gRkFMU0UpJT4lCiAgc2VwYXJhdGUoZHYsIGMoJ3Rhc2tfbmFtZScsICdleHRyYV8xJywgJ2V4dHJhXzInKSwgc2VwID0gJ1xcLicscmVtb3ZlPUZBTFNFKSAlPiUKICBzZWxlY3QoLWV4dHJhXzEsIC1leHRyYV8yKSAlPiUKICBnZ3Bsb3QoYWVzKG51bV9hbGxfdHJpYWxzLCBpY2MpKSsKICBnZW9tX3BvaW50KCkrCiAgZ2VvbV9zbW9vdGgobWV0aG9kPSJsbSIpKwogIHRoZW1lX2J3KCkrCiAgeGxhYigiTnVtYmVyIG9mIHRyaWFscyIpKwogIHlsYWIoIklDQyIpCmBgYAo=